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A  COMPARISON  OF  THE  USE  OF  DIRECT  AND  INDIRECT 
PHRASING  IN  PERSONALITY  QUESTIONNAIRES 

Chapter  I 
Introduction 


When  Woodworth  (28)  constructed 
the  Personal  Data  Sheet  during  the 
first  World  War,  he  started  a  fashion  in 
personality  questionnaires  which  has 
had  an  enormous  influence  on  psycho- 
logical and  educational  research  and 
practice.  In  the  course  of  the  last  thirty 
years,  the  popularity  of  paper  and  pencil 
personality  tests  has  been  impressively 
proved  not  only  by  their  widespread  use 
but  also  by  the  great  number  of  studies 
devoted  to  their  construction,  stand- 
ardization, reliability  and  validity. 

One  consequence  of  the  extensive  re- 
searches on  conventional  personality 
questionnaires  has  been  the  develop- 
ment, today,  of  instruments  which  are 
usually  conceded  to  be  highly  reliable. 
In  terms  of  validity,  however,  results 
have  been  far  less  decisive  and  promis- 
ing. Material  on  the  outcome  of  studies 
of  the  validity  of  personality  inventories 
has  been  reviewed  recently  by  the  present 
author  (4,  5),  who  summarized  the  valida- 
tion data  published  between  1919  and 
1946  on  tests  of  the  Woodworth,  Bern- 
reuter,  and  Thurstone  type.  The  find- 
ings were  as  follows: 

1.  When  used  to  distinguish  between 
groups  of  problem  and  nonproblem  chil- 
dren, personality  questionnaires  dis- 
criminated significantly  in  two  experi- 
ments, discriminated  with  a  questionable 
degree  of  significance  in  one,  and  were 
non-discriminative  in  six. 

2.  When  utilized  with  delinquents, 
the  questionnaires  differentiated  sig- 
nificantly between  delinquent  and  non- 
delinquent  groups  in  fifteen,  to  a  ques- 
tionable degree  in  six,   and   were   non- 


differentiating  in  thirteen  investigations. 

3.  When  employed  for  purposes  of 
psychiatric  and  psychological  diagnosis, 
the  inventories  were  adequate  for  group 
diagnosis  in  thirty-six,  questionably  ade- 
quate in  nine,  and  inadequate  in  twenty- 
two  studies. 

4.  When  scores  obtained  by  individ- 
uals taking  personality  questionnaires 
were  compared  to  ratings  given  these 
individuals  by  their  friends,  associates, 
teachers,  or  supervisors,  they  were  found 
to  be  closely  related  in  twelve,  related 
to  a  questionable  degree  in  ten,  and  to  a 
low  degree  in  twenty-two  investigations. 

5.  When  intercorrelations  between 
subjects'  scores  on  two  similar  personal- 
ity schedules  purporting  to  measure  the 
same  traits  were  studied,  they  were  found 
to  be  between  .70  and  1.00  in  nine,  be- 
tween .40  and  .69  in  eighteen,  and  be- 
tween .00  and  .39  in  twenty-eight  in- 
vestigations. 

6.  When  a  review  was  made  of  experi- 
ments specifically  designed  to  discover 
whether  subjects  taking  personality 
questionnaires  over-rated  themselves  or 
lied  about  their  responses,  it  was  discov- 
ered that  thirty-six  studies  indicated  that 
over-rating  did  take  place,  while  only  six 
showed  that  it  did  not. 

On  the  basis  of  these  findings,  it  was 
concluded  that  "group-administered  pa- 
per and  pencil  personality  questionnaires 
are  of  dubious  value  in  distinguishing 
between  groups  of  adjusted  and  malad- 
justed individuals,  and  .  .  .  are  of  much 
less  value  in  the  diagnosis  of  individual 
adjustment  or  personality  traits"  (4,  p. 
426). 
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Personality  inventories,  however,  havi 
definite  practical  advantages.  They  are 
eas)  to  administer  and  score;  give  results 
that  may  be  objectively  and  statistically 
treated;  ma)  serve  as  a  formalized  inter- 
view; and  at  least  give  a  picture  of  whai 
the  subjects  say  the}  Eeel  and  do.  Because 
of  these  advantages,  personality  tests 
have  maintained  their  popularity  despite 
the  recent  growth  of  projective  and  non- 
paper-and-pencil  techniques  of  assessing 
personality. 

Is  it  possihle  to  re-fashion  personality 
questionnaires  to  eliminate  some  of  their 
worst  drawbacks  and  yet  retain  the  ad- 
vantages of  ease  of  administration,  scor- 
ing and  interpretation?  Several  attempts 
in  this  direction  have  been  made  since 
Woodworth's  original  Personal  Data 
Sheet  was  published.  Thus,  Laird  (12) 
modified  the  YES-NO  method  of  re- 
sponse by  providing  for  several  possible 
answers  to  each  question.  Bernreuter 
(3)  introduced  the  inventory  method  of 
scoring  a  subject's  answers  to  a  single  set 
of  questions  so  that  he  receives  ratings 
on  several  different  traits.  Mailer  (14) 
suggested  stating  the  questions  in  a  less 
personal  form.  Humm  and  Wadsworth 
(9),  Washburne  (27),  and  McKinley  and 
Hathaway  (7,  15)  developed  methods  of 
estimating  dishonesty  and  correcting  for 
it.  McKinley  and  Hathaway  (7)  employed 
a  card-sorting,  individual  administrating, 
instead  of  the  usual  pencil-marking 
group  administration.  Adams  and  Lepley 
(1)  favored  innocuous  questions  that  pre- 
sumably would  not  embarrass  respond- 
ents. Shipley  and  his  associates  (19)  used 
a  forced  choice  technique  in  the  Per- 
sonal Inventory. 

All  the  foregoing  modifications  of  the 
original  Personal  Data  Sheet  have  added 
to  the  usefulness  of  personality  question- 
naires, but  none  has  yet  resulted  in  an 
instrument    whose    validity    is    unques- 


tioned. Further  experimentation  in  this 
connection  would  consequently  seem  de- 
sirable at  present. 

SPECIFIC    PURPOSES    OF    THE    PRESENT 
EXPERIMENT 

The  purpose  of  the  present  experiment 
was  to  study  the  effect  of  indirect  phras- 
ing in  personality  questionnaires.  The 
question  to  be  investigated  was:  Since 
the  doubtful  validity  of  conventional 
personality  schedules  may  be  partly  at- 
tributable to  respondents  often  discov- 
ering the  intent  of  the  questions,  deter- 
mining what  the  "good"  or  "correct" 
answers  are,  and  then  giving  "good"  re- 
sponses, is  it  possible  for  inventory  ques- 
tions to  be  indirectly  and  less  transpar- 
ently stated  so  that  schedules  using  them 
will  be  more  effective  for  diagnostic  and 
predictive  purposes  than  are  the  present 
direct  test  forms? 

Attempts  to  employ  indirect  ques- 
tions in  personality  schedules  are  not  en- 
tirely new.  Sweet  (23),  Spencer  (22),  and 
Tuckman  (26)  devised  questionnaires 
which  included  some  indirectly  phrased 
items.  None  of  these  investigators,  how- 
ever, specifically  tried  to  compare  the 
efficacy  of  direct  and  indirect  personality 
questions  utilized  on  the  same  subjects. 
The  point  of  departure  of  the  present 
experiment  was  that  it  studied  the  use 
of  equated  indirect  and  direct  person- 
ality items  in  the  same  test  battery;  in- 
vestigated the  effect  of  these  two  kinds 
of  personality  questions  on  a  behavior 
problem  and  a  non-problem  group  of 
children;  and  tried  to  determine  what 
was  the  effect  of  giving  the  direct  and 
indirect  questions  in  three  different  or- 
ders of  presentation— a  direct-indirect, 
alternate,  and  indirect-direct  order. 
There  were  thus  three  major  variables 
operating  in  the  experiment:  two 
groups;  two  kinds  of  tests;  and  three  dif- 
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fcrcnt  orders  of  presentation. 

The  specific  hypothesis  of  the  experi- 
ment was:  If  conventional  direct  per- 
sonality test  forms,  as  well  as  indirect 
schedules  derived  from  the  same  ques- 
tions included  on  the  direct  forms,  are 
given  in  a  single  battery  to  behavior 
problem  and  non-problem  children:  (1) 
the  indirect  test  scores  will  be  signifi- 
cantly different  from  the  direct  test 
scores;  (2)  groups  of  problem  and  non- 
problem  children  will  react  differently 
to  the  indirect  and  direct  tests;  and  (3) 
presentation   of   the   questions   in   three 


different  orders— an  indirect-direct,  an 
alternate,  and  a  direct-indirect  order- 
will  have  a  significant  effect  on  the  de- 
gree of  test  sophistication  shown  by  the 
children  in  their  responses. 

Another  way  of  stating  the  major 
premise  of  the  experiment  is  to  say  that, 
under  the  given  testing  conditions,  mak- 
ing use  of  the  three  major  variables  of 
tests,  groups,  and  orders,  it  is  hypothe- 
sized that  significant  differences  in  the 
subjects'  mean  test  scores  will  result  from 
the  influence  of  every  one  of  these  vari- 
ables. 


Chapter  II 
Experimental  Materials,  Subjects  and  Design 


MATERIALS 

In  order  to  obtain  a  schedule  of  direct 
questions  that  would  be  representa- 
tive of  the  conventional  tests  widely 
used  with  children  today,  a  frequency 
count  was  made  of  the  items  currently 
employed  in  the  Woodworth-Mathews 
(28),  Loofbourow-Keys  (13),  California 
(25),  Pintner  Aspects  of  Personality  (17), 
Rogers  (18),  Hildreth  (8),  Mailer  (14), 
and  Symonds  (24)  personality  inven- 
tories. The  thirty-six  most  popular  ques- 
tions were  culled  from  these  tests  and 
were  used  as  the  basic  direct  question- 
naire (Test  D-i)  employed  in  the  experi- 
ment. (The  complete  questionnaire  is 
printed  in  the  Appendix.) 

In  order  to  see  whether  the  manner  of 
phrasing  the  direct  questions  them- 
selves was  of  significant  import,  a  second 
direct  form  (Test  D-2)  was  constructed. 
In  Test  D-2  the  same  thirty-six  ques- 
tions used  in  Test  D-i  were  rephrased  so 
that  while  D-i  items  asked  about  a  nega- 
tive symptom,  D-2  items  asked  about  a 
positive  one.  Thus,  where  the  third  ques- 
tion on  D-i  read,  "I  cry  VERY 
OFTEN  PRETTY  OFTEN  SELDOM 
NEVER,"  the  equivalent  item  on  D-2 
read,  "I  am  free  from  crying  VERY 
OFTEN  PRETTY  OFTEN  SELDOM 
NEVER."  Whereas  Test  D-i  was  scored 
so  that  the  subjects  received  one  point 
for  each  item  answered  VERY  OFTEN 
or  PRETTY  OFTEN  and  no  points  for 
each  one  answered  SELDOM  or  NEVER, 
Test  D-2  was  scored  so  that  subjects  re- 
ceived one  point  for  each  SELDOM  or 
NEVER  response  and  no  points  for  each 
VERY  OFTEN  or  PRETTY  OFTEN 
answer. 

The  first  of  the  two  indirect  test  forms 


used  in  the  experiment  (Test  I-i)  was 
constructed  by  taking  the  same  thirty- 
six  questions  utilized  in  Test  D-i  and 
rephrasing  them  so  that  the  subjects  were 
asked  to  report  how  queer  they  consid- 
ered other  children  who  had  symptoms 
like  those  described  in  the  D-i  items. 
Thus,  the  I-i  version  of  the  question  on 
crying  was,  "Children  who  often  cry  are 
VERY  QUEER  PRETTY  QUEER  A 
LITTLE  QUEER  NOT  AT  ALL 
QUEER."  The  theory  behind  this  form 
of  indirect  questioning  was  that  chil- 
dren possessing  a  symptom  like  frequent 
crying  would  tend  to  see  other  children 
who  had  this  symptom  as  being  A  LIT- 
TLE QUEER  or  NOT  AT  ALL 
QUEER  rather  than  as  VERY  QUEER 
or  PRETTY  QUEER.  Preliminary  ex- 
perimentation showed,  however,  that  the 
I-i  items  were  more  meaningfully  evalu- 
ated when  a  score  of  one  point  was  given 
to  each  NOT  AT  ALL  QUEER  response 
and  a  score  of  no  points  to  the  VERY 
QUEER,  PRETTY  QUEER,  and  A 
LITTLE  QUEER  responses.  This,  con- 
sequently, was  the  scoring  system  em- 
ployed for  Test  I-i. 

The  second  indirect  questionnaire  em- 
ployed in  the  experiment  (Test  I-2)  was 
constructed  by  using,  once  again,  the 
original  thirty-six  questions  of  Test  D-i, 
but  this  time  asking  the  subjects  how 
many  other  children  they  thought  pos- 
sessed the  symptoms  described  in  the  D-i 
items.  Thus,  where  on  D-t  the  subject 
was  asked  how  often  he  cried,  on  the 
equivalent  1-2  item  he  was  asked  to  re- 
spond to  the  question,  "VERY  MANY 
PRETTY  MANY  FEW  NO  children 
often  cry."  The  theory  behind  the  1-2 
rephrasings  of  the  D-i  items  was  that  the 
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subje<  ts  who  actually  possessed  symptoms 
like  crying  frequently  would  tend  to  re- 
port that  there  were  VERY  MANY  or 
PRE  I  1  Y  MANY  children  who  also  had 
these  symptoms;  while  subjects  who  ac- 
tually exhibited  the  symptoms  seldom 
or  never  would  tend  to  report  FEW  or 
NO  other  children  as  having  them.  Con- 
sequently, on  1-2  items  a  score  of  one 
point  was  given  for  the  VERY  MANY 
or  PRETTY  MANY  responses  and  a 
score  of  no  points  for  the  FEW  or  NO 
answers. 

All  four  test  forms,  D-i,  D-2,  I-i  and 
1-2,  were  bound  together  in  a  single 
booklet  and  administered  to  the  sub- 
jects at  one  sitting,  in  the  manner  ex- 
plained below.  The  only  additional  ma- 
terial employed  in  the  experiment  wras 
the  Pintner  Intermediate  Test  (16),  a 
standard  group  test  of  intelligence 
which  was  routinely  administered  to  the 
subjects  several  weeks  before  the  ad- 
ministration of  the  personality  tests. 

SUBJECTS 

The  subjects  of  the  experiment  con- 
sisted of  two  groups  of  boys  from  regular 
seventh  and  eighth  grade  classes  of  three 
New  York  City  junior  high  schools: 

i.  The  behavior  problem  children— of 
which  there  were  forty— were  known  as 
serious  problem  cases  in  their  schools. 
This  meant  that  each  pupil  had  been 
brought  to  the  attention  of  the  school 
principal  on  several  occasions  and  had 
been  referred  to  the  school  psychologist 
for  special  guidance.  While  known  as 
problem  children,  these  forty  subjects 
were  not  among  the  worst  problems  in 
the  New  York  City  schools,  and  were 
pel  mitted  to  pai  ticipate  in  regular  school 
classes  rather  than  special  or  segregated 
ones.  There  is  little  reason  to  assume- 
that  the  problem  group  was  a  homo- 
geneous one,  since   the  offences  against 


school  discipline  lor  which  the  members 
ol  the  group  were  brought  to  the  atten- 
tion of  the  authorities  were  quite  varied. 
Moreover,  even  when  similar  offences 
were  committed,  the  reasons  l<>i  them 
may  have  ranged  from  so  ions  neurosis 
to  a  rather  healthy  aggressiveness. 

2.  The  non-problem  group  consisted 
of  221  unselected  school  children  in  regu- 
lar classes  who  were  having  no  special 
behavior  difficulties  in  school. 

The  test  booklets  were  administered 
to  the  children  by  their  own  teachers 
during  their  home-room  periods.  All  the 
children  in  each  class  were  tested,  the 
problem  along  with  the  non-problem 
subjects.  Segregation  of  the  two  groups 
of  papers  was  made  only  after  they  had 
been  handed  in  and  scored. 

DESIGN  OF  THE  EXPERIMENT 

In  designing  the  experiment,  several 

considerations  had  to  be  taken  into  ac- 
count: 

t.  The  four  personality  test  forms  had 
to  be  given  at  equally  spaced  time  in- 
tervals, and  at  intervals  which  would 
minimize  the  factor  of  intervening  oc- 
currences. This  was  accomplished  b\ 
binding  them  together  in  a  single  test 
booklet  which  could  be  answered  b\  all 
the  subjects  at  one  sitting.  Since  there 
were  only  144  brief  questions  all  told, 
the  element  of  test  fatigue  was  negligible. 

2.  The  tests  had  to  be  presented  in 
three  different  orders:  (a)  all  the  indireel 
questions  followed  by  all  the  elite  et  ones: 
(b)  a  page  of  indirect  alternated  with  a 
page  of  direct  questions:  and  (c)  all  tin 
direct  questions  followed  by  all  the  in- 
direct ones.  This  was  done  by  dividing 
each  test  into  four  pages,  each  page  con- 
taining nine  printed  questions.  J  he 
pages  were  then  assembled  and  bound  in 
the  three  required  orders,  so  that  on< 
third  of  the  children  could  In-  given  in- 
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direct-direct,    one-third    alternate,    and 
one-third  direct-indirect  booklets. 

•;.  Perseveration  tendencies  on  the 
pari  of  the  subjects  had  to  be  minimized. 
1  his  was  accomplished  by  assembling 
the  sixteen  pages  of  each  booklet  so  that 
two  pages  of  the  same  test  form  never 
Eollowed  each  other.  Thus,  in  the  in- 
direct-direct booklets,  there  would  be  a 
page  of  I--'  questions  alternated  with  a 
page  of  It  questions  lor  the  first  hall 
ol'  the  battery;  and  a  page  of  D-i  alter- 
nated with  a  page  of  D-a  questions  for 
the  last  hall  of  the  booklet. 

.].  Randomization  of  the  individual 
questions  in  each  test  had  to  be  effected. 
This  was  done  by  numbering  them  on 
separate  slips  of  paper,  mixing  them 
thoroughly  in  a  box,  and  then  picking 
them  out  individually  and  printing 
them  in  the  order  thus  selected. 

5.  Randomization  of  the  individuals 
and  equalization  of  the  numbers  of  sub- 
jects taking  the  three  test  orders  had  to 
be  arranged.  This  was  accomplished  by 
stacking  the  booklets  so  that  an  indirect- 
direct  battery  was  followed  by  an  aler- 
nate  one,  which  was  in  turn  followed  by 
a  direct-indirect  one,  and  so  on.  The 
booklets  were  then  distributed  to  the 
subjects  so  that  chance  factors  deter- 
mined which  test  order  any  individual 
child  received;  while,  in  every  class 
tested,  approximately  one-third  of  the 
children  received  a  direct-indirect,  one- 
third   an   alternative,   and  one-third   an 


indirect-direct  booklet.  The  fact  that 
children  who  sat  near  each  other  had 
diferently  arranged  booklets  also  meant 
that  attempts  at  copying  were  frustrated. 

6.  Environmental  factors  in  the  school 
situation  had  to  be  similar  for  both 
problem  and  non-problem  subjects  in 
order  to  effect  maximal  experimental 
control.  This  was  done  (as  outlined 
above)  by  selecting  problem  children 
who  attended  regular  school  classes  and 
who  were  therefore  not  responding  un- 
der segregated  or  other  conditions  which 
might  have  effected  the  testing  environ- 
ment. 

7.  Randomization  of  individuals  and 
equalization  of  numbers  of  subjects  tak- 
ing the  three  test  orders  had  to  be  ac- 
complished for  problem  children  as  well 
as  non-problem  children  involved  in  the 
experiment.  This  was  arranged  by  hav- 
ing the  problem  subjects  answer  the  test 
battery  in  their  regular  classroom  groups, 
exactly  as  did  the  non-problem  children; 
so  that  no  distinction  was  made  between 
the  two  groups  in  the  course  of  the  test 
administration. 

In  brief:  the  experiment  was  so  de- 
signed as  to  permit  the  four  personality 
test  forms  to  be  given  to  the  two  ex- 
perimental groups  in  three  different 
orders  of  presentation  without  any  sys- 
tematic biases  as  to  the  arrangement  of 
the  questions,  the  subjects,  the  tests,  the 
groups,  or  the  orders. 


Chapter  III 
Tin:  Data  and  Their  Treatmi     i 


The  basic  data  on  the  mean  scores  and 
variability  obtained  by  221  non- 
problem  and  40  problem  children  on 
the  four  personality  test  forms  used  in 
the  experiment  are  presented  in  Table  1. 
From  Table  1  it  would  appear  that, 
for  both  the  problem  and  non-problem 
groups:    (1)    Tests   I-i    and    1-2    differed 


(20,  21),  seemed  to  be  the  most  practi<  al 
statistical  method  of  treating  the  data 
of  Table  1.  Application  ol  Bartlett's  (2) 
test  for  the  homogeneity  of  variance  to 
the  data  revealed  a  sufficient  degree  of 
homogeneity  to  make  the  use  of  analysis 
of  variance  feasible,  so  it  was  accordingly 
employed. 


Table  i 

Mian  scores  and  standard  deviations.  221  non-problem  and  40  problem  children  taking  each 
of  four  forms  of  a  personality  test  in  three  orders  of  presentation 


Mea 

n  Score  and  Standard  Deviation 

Order  of 
Presentation 

Test  I- 

1 

Test  I- 

2 

Test  D 

-1 

Test  D 

-2 

N 

M 

S.D. 

N 

M 

S.D. 

N 

M 

S.D. 

N 

M 

S.D. 

No  -Problem 

Children 

Indirect- 

direct 

69 

15-73* 

7-44 

69 

13-33 

6-55 

69 

4.29 

3-92 

69 

3-75 

3-53 

Alternate 

78 

14-83 

8.70 

78 

11. 81 

6.88 

78 

4.41 

5-31 

78 

3-87 

3-64 

Direct- 

indirect 

74 

12.62 

8-43 

74 

10.00 

6.47 

74 

4-77 

6.47 

74 

4-35 

3-73 

Problem 

Children 

Indirect- 

direct 

13 

16.53 

6.07 

13 

12.46 

5" 

13 

6-54 

6.84 

13 

6.92 

4.96 

Alternate 

14 

17.21 

9.21 

14 

12.21 

4.71 

14 

6.64 

4-47 

14 

6.36 

512 

Direct- 

indirect 

13 

16.85 

10.32 

13 

8.69 

7-05 

13 

3.62 

3-65 

13 

4.07 

3-°4 

*  All  calculations  were  carried  out  to  the  third  decimal  place;  but  for  ease  of  reading,  only  two 
decimal  places  are  used  in  this  and  all  succeeding  tables. 


from  each  other;  (2)  Tests  D-i  and  D-2 
did  not  differ;  (3)  the  two  indirect  tests 
diflered  from  the  direct  ones;  and  (4) 
some  order  differences  existed.  The  ques- 
tion is:  Are  these  differences  statistically 
significant? 

Since  three  main  variables  were  in- 
volved in  the  experiment— groups,  tests, 
and  orders— and  since  considerable  inter- 
action among  these  variables  was  pos- 
sible, the  analysis  of  variance  technique, 
devised  by  Fisher  (6)  and  ably  ex- 
pounded   in    this    country   by    Snedecor 


THE    PROBLEMS    TREATED 

Problem  ia.  Were  the  subjects'  mean 
scores  on  the  four  personality  tests  sig- 
nificantly affected  by:  Form  of  the  test? 
Composition  of  the  group  taking  the 
test?  Order  of  presentation  of  the  ques- 
tions? 

Table  2  presents  the  basic  analysis  of 
variance  applied  to  the  data  of  Table  1. 
Table  2  may  be  interpreted  as  follows: 
If  the  Mean  Square  of  the  individual 
scores  within  subclasses  (37.09)  is  taken  as 
1  he  best  estimate  of  the  error  variance 
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Table  2 


Analysis  of  variance  of  mean  scores.  221  non-problem  and  40  problem  children  taking  each  of  four 
forms  of  a  personality  test  in  three  orders  ol  presentati<  n 


Source  of 
Variation 


I  Valves  Sum 

of  of 

Freedom       Squares 


Mean 
Square 


F, 


Tests 
Orders 

Interaction: 

Tests  X  Groups 
Interaction: 

Tests  X  Orders 
Interaction: 

Groups  X  Orders 
Interaction: 

Groups  X   Tests 
X  Orders 
Individual     scores 

class*  :s 

Total 


3 
1 
2 

3 
6 


within     suh- 


1020 


io4. 


21,095.01 
204.64 
397-38 

171.63 

222.19 

49 -40 

42S.52 

37,827.49 
60 , 396 . 86 


7,031.87 
204 . 64 
198.59 

57-21 

37-°3 
24.70 

71.42 
37-09 


189.61** 

5-52* 
5.36** 

1-54 
1  .00 

.67 
r-93 


2.61 

3-85 
3.00 

2 .61 

2 .  10 
3.00 


2 .  10 


3.80 
6.66 
4.62 

3.80 

2.82 
4  .62 

2.82 


*  Significant. 
**  Highly  significant. 

among  the  individuals  taking  the  four  test 
forms,  then  there  is  considerably  less 
than  one  chance  in  a  hundred  that  a 
difference  as  large  as  the  observed  dif- 
ference in  the  mean  score  among  the 
four  tests  would  be  caused  by  chance 
factors;  there  is  less  than  one  chance  in 
a  hundred  that  a  difference  as  large  as 
the  observed  difference  in  the  mean  score 
among  the  three  orders  would  be  so 
caused;  and  there  is  only  from  one  to 
five  chances  in  a  hundred  that  a  dif- 
ference as  large  as  the  observed  dif- 
ference in  the  mean  score  between  the 
two  groups  would  accidentally  occur. 
Stated  differently:  if  the  individual  score 
variation  is  taken  as  a  valid  estimation 
of  error  variance,  it  may  be  stated  with 
great  confidence  that  the  obtained  test 
and  order  differences,  and  with  reason- 
able confidence  that  the  observed  group 
differences,  did  not  arise  by  chance. 

If,  instead  of  using  the  variation 
among  individual  scores  as  the  best  esti- 
mate of  variance,  the  standard  error  of 
estimate  is  calculated  by  combining  the 


appropriate  mean  squares  of  first-order 
interactions,  second-order  interactions, 
and  individual  scores,  and  dividing  by 
their  combined  degrees  of  freedom,  the 
F-values  for  tests,  groups  and  orders 
respectively  become  188.37,  5-5^,  and 
5.32— which  are  almost  identical  with  the 
values  in  Table  2.  This  analysis  of 
variance,  evidently,  is  so  stable  that  the 
F-values  maintain  their  significance  even 
when  their  base  is  changed— a  fact  which 
should  enhance  confidence  in  the  sig- 
nificance of  the  observed  mean  differ- 
ences. 

Another  way  of  interpreting  the  three 
significant  F-values  of  Table  2  is  to  say 
that,  with  the  influence  of  groups  and 
orders  eliminated,  the  mean  scores  of  the 
four  tests  were  significantly  different. 
With  the  effect  of  tests  and  orders 
eliminated,  the  group  differences  were 
significant.  Finally,  with  effects  of  tests 
and  groups  eliminated,  order  of  presenta- 
tion differences  were  significant. 

Problem  ib.  Did  presentation  of  all 
four  test  forms  in  a  single  battery  pro- 
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(luce  significant  interaction  effects? 

Since  the  lust-  and  second-order  inter- 
action / -v  alucs  of  Table  2  arc  all  low. 
and  none  of  them  reaches  significance 
.n  even  the  V  ,  level  of  confidence,  there 
is  no  evidence  that  significant  degrees 
of  interaction  among  tests,  groups,  or 
orders  took  place. 

Problem  ic.  Were  there  significant 
differences  between  scores  obtained  on: 
The  first  and  second  direct  tests?  The 
first  and  second  indirect  tests?  The  direct 
and  indirect  tests? 

Table  2  must  be  analyzed  into  smaller 
segments  to  provide  more  detailed  con- 
clusions to  several  problems  of  the  ex- 
periment. Thus,  although  the  F-value 
of  189.61  for  tests  shows  that  the  lour 
tests  differed  significantly  among  them- 
selves, it  does  not  clearly  indicate 
whether  D-i  mean  scores  differed  from 
those  of  D-2,  I-i  from  1-2,  nor  D-i  and 
D-2  from  I- 1  and  1-2.  However,  if  the 
mi  an  square  for  tests  is  appropriately  sep- 
arated into  the  three  degrees  of  freedom 
of  which  it  is  comprised,  these  questions 
ma)  be-  answered: 

1.  Between  the  mean  scores  of  Tests 
D-i  and  D-2,  F-values  of  2.45  and  .02 
are  obtained,  respectively,  for  non-prob- 
lem and  problem  subjects.  Neither  of 
these  F-ratios  is  significant  at  the  .05  level 
of  confidence. 

2.  Between  Tests  I-i  and  I-2  highly 
significant  /'-ratios  of  49.07  and  65.0$ 
are  obtained,  respectively,  for  non-prob- 
lem and  problem  children.  Apparently, 
the  two  direct  tests  did  not  differ  si<-- 
nificantly  from  each  other,  but  the  two 
indirect  ones  did. 

3.  Between  the  two  direct  tests,  on  the 
one  hand,  and  the  two  indirect  sched- 
ules, on  the  other  hand,  /''-ratios  are 
found  to  be,  for  non-problem  and  prob- 
lem   children    respectively,     101.48    and 


8.05,  botli  of  which  are  highly  signifi- 
cant. Thus,  the  two  direct  tests  differed 
distinctly  from  the  two  indirect   forms. 

Problem  id.  Which  of  the  lour  tests 
significantly  differentiated  the  problem 
from  the  non-problem  groups  ol  sub- 
jects? 

While  Table  2  indicates  that  then 
were  significant  differences  between  the 
mean  scores  of  the  problem  and  non- 
problem  children,  it  does  not  specifi- 
cally show  which  of  the  four  tests  ac- 
counted for  these  differences.  Accord- 
ingly, the  data  were  arranged  so  that 
analyses  of  variance  could  be  done  by 
individual  tests.  These  anlayses  are  pre- 
sented in  Table  3 

From  Table  3  it  may  be  seen  that  the 
F-ratios  for  group  differences  for  Tests 
I-i  (3.08),  I-2  (.02)  and  D-i  (2.09)  are 
not  significant.  However,  that  of  Test 
D-2  (7.26)  is  quite  significant.  In  other 
words,  on  Tests  I-i,  1-2,  and  D-i  there 
is  a  good  chance  that  differences  as 
large  as  the  observed  differences  be- 
tween the  mean  scores  of  problem  and 
non-problem  children  could  have  arisen 
purely  by  accident.  But  there  is  less  than 
one  chance  in  a  hundred  that  the  ob- 
tained group  difference  on  Test  D-2  was 
an  accidental  one. 

Problem  ie.  Did  order  of  presentation 
of  the  questions  make  a  significant  dif- 
ference on  test  scores? 

Table  2  indicates  that  order  of  presen- 
tation of  the  questions  effected  signifi- 
cant differences  in  mean  test  scores. 
Table  3  indicates  that  when  problem 
and  non-problem  groups  were  considered 
together,  significant  order  differences  <><- 
curred  with  Test  1-2,  but  not  with 
Tests  I-i.  D-i,  and  D-2.  When  analyses 
ol  variance  are  performed  on  the  mean 
test  scores  with  groups  and  tests  con 
sidered  separately,  the  F-ratios  shown  in 
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Tablk  3 

Analysis  of  variance  of  mean  scores.  221  non-problem  and  40  problem  children  taking  each  of  four 
forms  dt  a  personality  test  in  three  orders  of  presentation 


Source  of 
Variation 

Decrees 
of 

Freedom 

Sum 
of 

Squares 

Mean 
Square 

F 

r.  01 

/••„, 

Test  I- 1 

t iroups 

1 

213-13 

213-13 

3.08 

3-89 

6.76 

Orders 

2 

312. 28 

156.14 

2.  26 

3  04 

4.71 

Interaction: 

1  rroups  X  Ord<  rs 

2 

59-65 

29-73 

•43 

3 -°4 

4-7i 

Individual  scores 

-'55 
260 

17,621.94 

69. 11 

Total 

18, 207.00 

Test  I-2 

Groups 

1 

9-47 

9-47 

.02 

3-89 

6.76 

Orders 

2 

495-77 

247-83 

5-5i** 

3-°4 

4.71 

Interaction: 

Groups  X  Orders 

2 

19.72 

9.86 

.  22 

3 -04 

4.71 

Individual  scores 

255 
260 

11,217.81 

43-99 

Total 

11,742.77 

Test  D-i 

Groups 

1 

43-39 

43-39 

2  .09 

3-89 

6.76 

Orders 

2 

1 .  12 

•56 

•03 

3-°4 

4-7i 

Interaction: 

Groups  X  Orders 

2 

85-81 

42.91 

2 .  16 

3-°4 

4.71 

Individual  scores 

255 

5,171.69 

20.28 

Total 

260 

5,302.01 

Test  D-2 

Groups 

1 

no.  29 

no.  29 

7.26** 

3-89 

6.76 

Orders 

2 

.82 

.41 

.02 

3-°4 

4.71 

Interaction: 

.    Groups  X  Orders 

2 

73-32 

36.66 

2  .42 

3-°4 

4.71 

Individual  scores 

255 
260 

3,865.26 

15.16 

Total 

4,029.69 

**  Highly  significant. 

Table  4  are  obtained.  In  these  analyses, 
the  variance  among  the  mean  scores  in  the 
three  orders  (indirect-direct,  alternate, 
and  direct-indirect)  are  compared  with 
the  estimate  of  error  variance  obtained 
by  considering  the  variation  among  the 
scores  of  the  individuals  taking  each  test 
order. 

If  Table  4  is  read  in  conjunction  with 
Table  1  the  following  points  may  be 
noted: 

1.  Non-problem  children  taking  Test 
I-i  obtained  a  higher  (that  is,  worse) 
mean  score  on  the  indirect-direct  (15.73) 


than  on  the  alternate  (1483)  or  the 
direct-indirect  (12.62)  orders  of  presenta- 
tion. This  increase  in  indirect-direct 
mean  score  is  statistically  significant. 

2.  Non-problem  children  taking  Test 
1-2  also  obtained  higher  scores  on  the 
indirect-direct  (13-33)  than  on  the  alter- 
nate (11.81)  or  the  direct-indirect  (10.00) 
orders.  This,  too,  is  a  statistically  signifi- 
cant increase. 

3.  Non-problem  children  taking  test 
D-i  obtained  a  lower  (or  better)  mean 
score  on  the  indirect-direct  (4.20)  than 
the  alternate  (4.41)  or  the  direct-indirect 
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1  1 


I   \BI.K  4 

Order  of  presentation   /-'Ratios  (order  of  pre- 
sentation variance  'error  variance).  221  non-prob- 
lem and  40  problem  children  taking  each  of  four 
forms  of  a  presonality  test  in  thru- 
orders  of  presentation 


( Observed 

Test 

Value 
of  F 

F.ui 

F.oi 

Non-Problem 

Children 

Test  I- 1 

3.09* 

3°4 

4-7i 

Test  1-2 

5  •  »o** 

3 -°4 

4.71 

Test  D-i 

2  .02 

3  °4 

4-7i 

Test  D-2 

2-34 

3  04 

4-71 

Problem 

Children 

Test  I- 1 

.02 

3-25 

5-2i 

Test  I-2 

.98 

3-25 

5-21 

Test  D-i 

1-45 

3-25 

5-21 

Test  D-2 

1-47 

3-25 

5-21 

*  Significant. 
**  Highly  significant . 


(4.77)    order.    This   decrease   is   not   sta- 
tistically significant. 

4.  Non-problem  children  taking  test 
D-2  again  obtained  a  lower  mean  score 
on  the  indirect-direct  (3.75)  than  on  the 
alternate  (3.87)  or  the  direct-indirect 
(4.35)  order  of  presentation.  This,  too, 
is  not  a  significant  difference. 

5.  Problem  children  taking  the  I-i 
test  obtained  a  lower  mean  score  on  the 
indirect-direct  (16.54)  than  on  the  alter- 
nate (1721)  or  the  direct-indirect  (16.85) 
order.  But  problem  children  takingTests 
1-2,  D-i,  and  D-2  obtained  higher  mean 
scores  on  the  indirect-direct  than  they 
generally  did  on  the  alternate  and  direct- 
indirect  orders  of  presentation.  How- 
ever, none  of  these  differences  are  sig- 
nificant. 

The  data  of  1  ables  1  and  4  seem  to 
confirm  the  hypothesis  of  test  sophistica- 
tion. For  when  the  direct  questions  were 
printed  prior  to  the  incline  t  ones,  non- 
problem  subjects  made  significantly 
lower,  or  better,  scores  on  both  incline  1 


tests.  But  when  the  indirect  questions 
were  printed  first,  subjects  made  some- 
what (though  not  statistically  reliable) 
lower,  or  better,  scores  on  both  direct 
tests.  It  would  seem  that  presenting  the 
direct  questions  prior  to  the  indirect  ones 
enabled  non-problem  children  to  dis- 
cover the  intent  of  the  latter  and  answer 
them  in  a  favorable  manner.  Presenting 
indirect  questions  prior  to  direct  ones 
also  produced  a  degree  of  test  sophisti- 
cation on  the  latter,  but  not  enough  to 
make  the  mean  score  differences  statisti- 
cally significant.  The  fact  that,  in  all 
four  tests  taken  by  non-problem  chil- 
dren, the  mean  scores  on  the  alternate 
order  of  presentation  were  always  in  be- 
tween those  obtained  on  the  indirect- 
direct  and  direct-indirect  orders  lends 
additional  weight  to  the  theory  that  a 
measurable  degree  of  test  sophistication 
took  place. 

Concerning  the  behavior  problem  chil- 
dren, the  evidence  on  order  of  presenta- 
tion differences  is  conflicting  and  baf- 
fling. It  is  probable  that  since  so  few 
problem  subjects  were  employed  in  each 
sub-grouping,  and  since  the  sub-groups 
may  have  differed  widely  among  them- 
selves, the  observed  order  differences 
here— none  of  which  reaches  statistical 
significance— are  meaningless. 

Problem  2a.  How  did  the  internal 
consistence  or  reliability  of  the  four 
tests  affect  the  subjects'  scores? 

It  is  important  to  know  the  internal 
consistency  or  reliability  of  each  of  the 
four  tests  to  see  if  lack  of  test  reliability 
may  have  accounted  for  some  of  the 
observed  differences.  As  Jackson  and 
Ferguson  (km  have  pointed  out,  what 
does  or  does  not  constitute  a  true  test 
reliability  is  still  a  matter  of  controversy. 
Test-retest,  split-half,  internal  consis 
tency  and  other  methods  commonly  used 
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[or  obtaining  reliabilities  each  has  it ^ 
advantages  and  disadvantages.  In  the 
present  study,  the  K.uder-Ri<  hardson  (i  \) 
formula  For  measuring  internal  consis- 
tency was  employed  -because  the  lour 
tesl  forms  were  administered  onlj  once 
and  because  this  formula  gives  .1  single 
unique  estimate  ol  internal  consistency 
or  reliability. 

Table  5  presents  the  internal  con- 
sistency quotients  obtained  1>\  applying 
the    Kuder-Richardson   formula    No.    20 


interaction  among  tests,  groups,  and 
assemblies  as  the  error  of  estimate,  non- 
significant /'ratios  were  found  lor  tests 
(.92),  groups  (.18)  and  orders  (.22).*  It 
would  seem,  therefore,  that  there  were 
no  significant  test,  group  or  order  dif- 
ferences in  the  internal  consistency  quo- 
tients. Apparently,  the  observed  mean 
score  differences  were  not  attributable  to 
differences  in  the  internal  consistency  ol 
thi'  lour  test  forms. 

Problem    2h.    Were    the    obtained    dif- 


Table  5 

Internalconsistency  or  reliability  coefficients  (Jxuder-Richardson).  221  non-problem  and  40  problem 
children  taking  each  of  four  forms  of  a  personality  test  in  three  orders  of  presentation 


Test 


Internal  Consistency  Quotient 
of  Non-Problem  Children 

Order  of  Presentation 


Internal  Consistency  Quotient 
of  Problem  Children 

Order  of  Presentation 


Indirect- 
Direct 

(N  =  6o) 


Alternate 
(N  =  78) 


Direct- 
Indirect 
(N  =  74) 


Indirect- 
Direct 

(N  =  i3) 


Alternate 
(N  =  i4) 


Direct- 
Indirect 

(N  =  i3) 


I-I 

.90 

.92 

.86 

.76 

.91 

•93 

1-2 

.89 

.90 

.76 

.70 

/-\  iy  *£  •>'  * 

.88 

D-i 

.82 

.91 

•75 

•94 

•83 

■75 

D-2 

.78 

•79 

•78 

.84 

•94 

. 50*** 

*  All  the  correlations  in  the  table  are  statistically  significant  from  zero  at  the  1%  level  of  confi- 
dence except  the  two  starred  ones. 


to  the  original  score  data  of  the  experi- 
ment. It  can  be  seen  that  while  none  of 
the  correlations  in  Table  5  is  exception- 
ally high,  those  of  the  non-problem  chil- 
dren are  all  significantly  different  from 
zero  at  the  1%  level  of  confidence,  ac- 
cording to  Fisher's  (6)  test  for  the  signifi- 
cance of  correlations.  Those  of  the  prob- 
lem children,  how'ever,  are  not  as  high 
as  those  of  the  non-problem  subjects. 

The  internal  consistency  quotients  in 
Table  5  may  be  converted  into  z-values 
(Fisher's  transformation)  and  tested  for 
significant  differences  among  them. 
When  this  was  done,  and  an  analysis 
of  variance  performed  upon  the  z-values 
thus    obtained,    using    the    second-order 


ferences  in  mean  scores  attributable  to 
lack  of  item  stability  on  the  four  tests? 
To  discover  if  the  tests  possessed  a 
sufficient  degree  of  item  stability  to  war- 
rant the  assumption  that  the  obtained 
test,  group,  and  order  mean  score  dif- 
ferences were  not  merely  caused  by  the 
use  of  unstable  testing  instruments,  an 
item  analysis  was  done  of  each  of  the 
four   tests  used  with   non-problem  chil- 

*  This  analysis  of  variance  of  the  rvalues,  as 
well  as  those  made  in  the  next  few  pages,  is 
based  on  the  assumption  of  equal  reliability  for 
the  original  correlations  and  therefore  is  not  to 
be  viewed  as  too  precise  a  test  of  significance  of 
test,  group,  and  order  differences.  While  it  is  the 
best  that  can  be  made  under  the  circumstances, 
its  results  should  be  taken  as  approximations 
rather  than  as  exact  determinations. 


PHRASING    IN     PFKS(>\\LITY    QUESTIONNAIRES 


13 


dren.  A  similar  analysis  was  not  done  lor 
the    problem    children's    responses    be 
(.ins-     the    small    number    of    problem 
children    employed    in    the    experiment 
made  it  impractical. 

In    the    item    stability    analysis,     the 


biserial  correlation  was  found  between 
toial  test  score  and  each  item  response. 
The  obtained  correlations,  classified  ac- 
cording 10  order  ol  presentation  ol  the 
tests,  are  shown  in  Table  <>.  An  exam- 
ination of  Table  6  reveals  the  following: 


Table  6 

Bi-serial  correlations  between  responses  to  individual  test  items  and  total  test  scores. 
221  non-problem  children  taking  each  of  four  forms  of  a  personality  t( 
in  three  different  orders  of  presentation 


Item 


Bi-serial  Correlation  Coefficient  Between  Item  and  Total  Score 


Test  1-. 


Test  1-2 


Test  D-i 


Test  D-2 


I-D"       ALTb     D-Ic        I-D       ALT    D-I 


I-D       ALT    D-I 


I-D       ALT     D-I 


I 

2 

•30 
.48 

3 

•75 

4 

•59 

6 

7 
8 

•72 
.69 
•38 
■S3 

9 

■49 

IC 

•5° 

11 

.28 

12 

•43 

13 

•45 
.69 

15 
16 

•74 
.62 

17 
18 

•73 
•29 

19 

•73 

20 

•54 

21 

.48 

22 

•52 

23 
24 

•55 
.61 

25 
26 

•5° 
•35 

27 

•70 

28 

•59 

29 

•59 

3° 

•39 

3i 
32 

.67 
.58 

33 
34 

•59 

.60 

35 
36 

•57 
.58 

\\<rage 


58 


8; 
85 
6i 

89 

74 
88 
80 

49 

57 
55 

63 
9i 
64 
64 
82 

55 
72 
40 
71 
3° 

86 
89 

74 
86 

63 
73 
76 

*7 
7i 
55 

46 

5° 
76 

77 
35 
46 

•7i 


57 

.40 

79 
89 
88 

•32 
•50 
•52 

81 

■53 

92 
86 

■77 
.68 

82 
88 

87 

■39 
•45 
.66 

63 

•50 

9i 

78 

•59 

.82 

71 
68 
89 

•44 
•32 
•56 

92 

63 

96 

47 

■42 

•43 
.76 

.78 

89 

.92 

79 
88 

83 
66 

•55 
•45 
.64 
.61 

72 

.60 

70 

.92 

72 

•92 

79 

•7i 

24 

■33 

76 

•74 

57 

.  21 

53 
62 

68 

•53 
■85 
•7i 

55 

•39 

78 


90 
42 

38 

6< 

78 
35 
65 
7i 
68 

78 

50 
49 
39 

48 
81 

71 
48 
08 

83 
16 

53 
43 
55 
89 

87 
44 
92 

43 
89 

40 

82 
40 
88 
43 
44 
62 


.63        .64 


70 
52 
58 
60 

°7 
81 

67 

73 
68 

63 

65 
5  7 
43 
67 
5  7 
88 
82 
58 
69 
77 

74 
7i 

55 
60 

79 
52 

77 
70 
61 
54 

46 
48 
68 
64 

63 
60 

63 


88 
38 
42 

57 
37 
70 
78 
70 

15 
38 


,18 

•73 
.84 
.91 

■57 
■85 
•38 
.06 

•59 

.87 
.86 
.69 
.62 
•31 
•34 
•50 
•5i 

•  "I 
•39 

■59 
•37 
.92 

•37 
•23 
.  20 

•  54 


36 
90 

48 
41 
5  5 
85 
75 
94 
75 


83 
89 
60 
61 

93 
77 
89 
53 
45 
75 

90 
48 
92 
78 
94 
52 
95 
7° 
39 
66 

90 
42 
93 
57 
94 
61 

79 


53 
90 

38 
74 
81 

80 

44 
06 

75 
92 

86 

91 
27 

49 
1 1 

65 
58 
27 
15 
62 

93 
33 
58 
74 
09 

92 
42 

74 
53 
75 

58 
40 
7i 
85 

22 

5i 
.67 


•79 

•  M 
•50 
.41 
.92 
.48 
•50 

■3i 
.  22 

.46 


45 
53 
5i 
60 

29 
37 
49 
41 
59 

55 
58 
43 
5° 
48 
3° 
33 
'7 
20 

65 

27 
39 
80 

63 
37 
14 

.46 


77 
28 

74 
74 
88 
28 
92 

89 

75 

42 

9i 
4' 
88 
70 
9i 
93 
08 

70 

3i 
82 

9i 
60 
28 
55 
67 
1 1 

77 
70 
68 
40 

88 

79 
92 

52 

95 

40 

73 


63 
89 
4i 
69 

95 
41 
69 

4' 
95 
89 

88 
32 
46 

33 
88 

63 
59 
63 
50 
48 

72 
27 
84 
62 

79 
89 
19 
86 

3° 
18 

89 
48 

5  7 
73 
17 
47 

.68 


a  Indirect-direct  order  of  presentation. 
b  Alternate  order  of  presentation. 
c  Direct-indirect  order  of  presentation. 

''  Blank  spaces  indicate  that  there  were  no  responses  to  one  of  tin   categories,  all  the  individuals 
answering  the  item  in  one  way. 
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i.  All  the  correlations  are  positive— 
which  indicates  that  every  item  in  the 
loin  tests  contributed  something  to  the 
schedule  in  which  it  was  used.  Most  of 
the  correlations  are  above  .42,  and  are 
consequently  significantly  different  from 
ato  at  the  i'(,  level  of  confidence. 

2.  When  the  correlations  in  Table  6 
are  transformed  into  z-valiies  and  average 
correlations  calculated  for  each  order  of 
presentation,  the  averages  are  all  signifi- 
cantly different  from  zero  at  the  1% 
Level  of  confidence.  Evidently,  then,  the 
tests  employed  in  the  experiment  were 
not  composed  of  haphazardly  thrown  to- 
gether  items,  but  did  possess  some  degree 
of  item  stability. 

The  biserial  correlations  in  Table  6 
may  be  transformed  back  into  z-values 
and  analyzed  for  significant  differences 
among  them.  When  this  was  done,  using 
the  mean  square  of  the  individual  items 
as  the  estimate  of  error  variance,  analyses 
of  variance  produced  F-ratios  of  8.00 
for  tests  (3  and  420  degrees  of  freedom) 
and  62.00  for  orders  (2  and  420  degrees 
of  freedom),  both  of  which  are  highly 
significant.  That  is  to  say:  there  were 
significant  differences  among  both  tests 
and  orders,  thus  confirming  the  test  and 
order  mean  score  differences  previously 
discovered. 

Problem  5.  What  was  the  effect  of  the 
intelligence  quotients  of  the  subjects  on 
their  personality  test  scores? 

To  uncover  the  possible  effect  of  in- 
telligence quotients  on  personality  test 
scores,  the  Pintner  Intermediate  Test  of 
Mental  Ability  was  given  to  the  subjects. 
On  this  test,  the  non-problem  children 
had  an  average  intelligence  quotient  of 
109.74  ana"  the  problem  children  of 
100.42.  However,  the  problem  children's 
mean  chronological  age  was  13.82  against 
the  non-problem  children's  mean  of 
12.84.  Mean  mental  ages  of  the  problem 


children  were  found  to  be,  for  the  in- 
direct-direct, alternate,  and  direct-in- 
direct orders  respectively,  13.85,  13.97 
and  13.86;  while  those  of  the  non-prob- 
lem subjects  for  the  same  respective 
orders,  were  14.11,  14.21,  and  13.90. 
When  tested  by  Chi-square  analysis, 
these  were  not  found  to  be  significantly 
different,  since  Chi-square  equalled  only 
.84   for  two  degrees  of  freedom. 

The  correlations  between  the  intelli- 
gence quotients  of  the  individual  chil- 
dren and  their  scores  on  the  four  tests 
are  shown  in  Table  7.  Table  7  indicates 
that  whereas  in  many  instances  the  non- 
problem  children  showed  a  significant 
negative  degree  of  relationship  between 
their  intelligence  quotients  and  test 
scores  (that  is,  the  higher  their  intelli- 
gence, the  lower  and  more  favorable 
their  scores),  there  was  only  one  notable 
instance  of  this  occurring  with  the  prob- 
lem children. 

To  test  the  significance  of  the  group, 
test,  and  order  differences  in  the  correla- 
tions listed  in  Table  7,  these  were  con- 
verted into  z-values  and  an  analysis  of 
variance  performed.  Using  the  mean 
square  of  the  second-order  interaction 
(tests  X  groups  X  orders)  as  the  estimate 
of  error  variance,  these  F-ratios  were 
found:  for  groups,  4.11  (1  and  6  degrees 
of  freedom);  for  tests,  1.55  (3  and  6 
degrees  of  freedom);  and  for  orders,  2.84 
(2  and  6  degrees  of  freedom).  All  these 
F-ratios  are  non-significant,  indicating 
that  the  observed  mean  score  differences 
among  tests,  groups,  and  orders  probably 
were  not  caused  by  differential  intelli- 
gence quotient  effects. 

Problem  4.  What  were  the  intercor- 
relations  between:  The  direct  and  in- 
direct tests?  The  two  direct  tests?  The 
two  indirect  tests? 

The  intercorrelations  between  the 
scores  of  non-problem  and  problem  chil- 
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Table  7 

Correlation  coefficients  between  intelligence  quotients  and  scores.  221  non-problem  and  40  problem 
children  taking  each  of  four  forms  of  a  personality  test  in  three  orders  of  presentation 


Relationship 


Correlation  Coefficient  of 
Non-Problem  Children 

Order  of  Presentation 

Indirect-      Alter-        Direct- 
Direct  nate        Indirect 
(\=6q)     (N  =  78)     (N  =  74) 


Correlation  Coefficient  of 
Problem  Children 

Order  of  Presentation 

Indirect-      Alter-        Direct- 
Din.  1  nate         Indirect 
(N  =  i3)     (N  =  i4)     (N  =  i3) 


Intelligence  Quotients  and  I-i  Test 

Scores 

.04 

-.41** 

-•43** 

•25 

•44 

.09 

Intelligence  Quotients  and  1-2  Test 

Scores 

-.26* 

-•49** 

-.64** 

.26 

—  .  22 

-.66* 

Intelligence  Quotients  and  D-i  Test 

Scon  s 

—  A  A** 

—  .44 

.OI 

-.31** 

—  .02 

.14 

.16 

Intelligence  Quotients  and  D-2  Test 

Scores 

.06 

-    .14 

-•47** 

.  10 

.06 

-•45 

*  Significantly  different  from  zero  at  the  5%  level  of  confidence. 
"*  Significantly  different  from  zero  at  the  1%  level  of  confidence. 


(lien  on  all  four  test  forms  are  presented 
in  Table  IS.  From  an  examination  of 
Table  8  these  points  may  be  noted: 

1.  The  two  indirect  tests,  except  for 
the  indirect-direct  order  taken  by  the 
non-problem  children,  showed  only  a 
low  degree  of  intercorrelation. 

2.  The  degree  of  correlation  between 
the  indirect  and  direct  tests  was  not  very 
high  for  either  the  problem  or  non- 
problem  subjects. 

3.  The  degree  of  relationship  between 


Tests  D-i  and  D-2  was  reasonably  high 
for  the  non-problem,  but  not  for  the 
problem  children. 

4.  The  indirect-direct  order  of  presen- 
tation generally  resulted  in  higher  test 
intercorrelations  than  did  the  direct-in- 
direct order. 

When  the  correlations  in  Table  8  were 
converted  to  z-values  and  an  analysis 
of  variance  done  upon  them,  using  the 
mean  square  of  the  second-order  inter- 
action as  estimated  error  variance,  these 


Table  8 

Intercorrelation  coefficients  for  four  personality  tests  taken  by  221  non-problem  and 
40  problem  children  in  three  orders  of  presentation 


Relationship 


Intercorrelation  Coefficient 
of  Non-Problem  Children 

Order  of  Presentation 


Intercorrelation  Coefficients 
of  Problem  Children 

Order  of  Presentation 


Indirect- 

Alter- 

Direct- 

Indirect- 

Alter- 

Direct - 

Direct 

nate 

ed 

irect 

Direct 

nate 

Indirect 

(N=6o) 

(N  =  78) 

(N  = 

=74) 

(N  =  i3) 

(N  =  i4) 

(N  =  i3) 

Tests  I-i  and  I-2 

•75** 

.  12 



.09 

•47 

-•23 

•  17 

Tests  I-i  and  D-i 

—  .01 

•17 

•33** 

•54 

•  25 

-•23 

Tests  I-i  and  D-2 

.09 

.19 

.26* 

.66* 

.16 

-.40 

Tests  I-2  and  D-i 

.24* 

.48** 

.  22 

.66* 

■63* 

•35 

Tests  1-2  and  D-2 

•37** 

.52** 

.04 

.61* 

.71** 

•3i 

Tests  D-i  and  D-2 

.83** 

•77** 

.67** 

.97** 

•57* 

•38 

*  Significantly  different  from  zero  at  the  5%  level  of  confidence. 
**  Significantly  different  from  zero  at  the  1%  level  of  confidence. 
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/'-ratios  were  obtained:  relationships, 
4.13  (5  .ind  10  d.f.);  groups,  .,■,<>  (1  and  10 
d.f.);  and  orders,  6.95  (a  and  10  d.f.). 
These  F-ratios  indicate'  significant  ordei 
differences  at  the  5%  level  of  confidence; 
but  non-significant  group  differences. 

Problem    ^.   How  significantly  did   the 
items   on    the    lour   tests  distinguish    be- 
tween   non-problem    and    problem    sub 
jectsi 

To  discover  which  test  items  distin- 
guished significantly  between  problem 
and  non-problem  groups,  an  item  analy- 
sis was  performed,  using  the  Chi-square 
method  as  modified  by  Yates  (29)  lor  use 
with  small  cell  figures.  Specifically,  for  the 
36  items  on  each  of  the  tests,  the  number 
of  non-problem  and  problem  subjects 
answering  the  item  positively  and  nega- 
tively were  recorded  in  two-by-two  tables. 
A  Chi-square  test  for  homogeneity  of 
each  two-by-two  table  was  then  made.  If 
a  sufficiently  large  Chi-square  figure 
(representing  a  probability  of  .05  or  less) 
was  obtained,  it  was  assumed  that  homo- 
geneity did  not  exist  between  the  re- 
sponses of  the  two  groups  of  subjects, 
and  the  item  was  considered  to  distin- 
guish significantly  between  them. 

Table  9  lists  Chi-square  values  only 
for  those  items  which  differentiated  sig- 
nificantly between  non-problem  and 
problem  children  on  the  four  tests  em- 
ployed in  the  experiment,  classified  ac- 
cording to  order  of  presentation.  For  the 
purposes  of  readability,  all  insignificant 
Chi-square  values  are  omitted  from  the 
table. 

Observation  of  Table  9  reveals  the 
following: 

1.  Not  one  of  the  four  tests  contained 
many  items  which  significantly  differ- 
entiated non-problem  from  problem  chil- 
dren. Out  of  108  possible  items  for  each 
test  (36  test  items  given  in  three  differ- 
ent orders),  Test  I-i  included  only  12  sig- 


nificant ones;  Test  1-2,  4;  Test  D-i,  10; 
and  Test  D-2,  17.  Statistically  considered, 
this  is  .1  meagre  showing  since  about  five 
items  of  each  test,  on  a  chance  basis 
alone,  should  be  expected  to  show  Chi- 
square  values  as  huge  as  those  tabulated 
lor  the  .05  level  of  confidence. 

2.  The  indirect  tests  included  fewer 
differentiating  items  than  did  the  direct 
icsts.  However,  when  the  number  of 
direct  and  indirect  lest  discriminations 
were  treated  by  Chi-square  (that  is,  Hi 
out  of  a  possible  216  compared  to  27  out 
of  216),  Chi-square  was  found  to  equal 
.002  for  one  degree  of  freedom.  The  dif- 
ference is  not  significant. 

3.  Only  in  five  out  of  fifteen  instances 
were  significant  direct  test  items  also  sig- 
nificant on  one  or  more  of  the  orders 
of  the  indirect  tests.  On  the  other  hand, 
seven  of  the  ten  significant  D-i  items 
were  also  significant  on  one  or  more 
orders  of  the  D-2  test.  This  substantiates 
previous  findings  that  the  two  direct 
tests  differed  from  the  indirect  ones  but 
were  similar  to  each  other. 

4.  Some  order  of  presentation  differ- 
ences are  evident  in  Table  9.  Test  I-i 
showed  only  one  significant  item  value 
for  its  indirect-direct  administration;  but 
as  many  as  eight  on  its  direct-indirect. 
Test  1-2  had  too  few  significant  items  in 
any  presentation  order  for  realistic  com- 
parison. Test  D-i  displayed  four  signifi- 
cant items  on  the  indirect-direct  order 
but  only  one  on  the  direct-indirect.  Test 
D-2  also  showed  a  drop  from  ten  signifi- 
cant items  on  the  indirect-direct  order 
to  two  on  the  direct-indirect. 

5.  Analysis  of  the  content  of  the  items 
which  were  discriminating  in  Table  9 
shows  no  prominent  patterning.  With 
the  direct  tests,  however,  there  is  an 
interesting  finding.  The  subject  mater 
of  the  eight  questions  which  discrimi- 
nated on  one  or  more  of  the  presentation 
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Table  9 

Chi-square  values  indicating  the  ability  of  individual  test  items  to  discriminate  significantly 

between  non-problem  and  problem  groups  of  children  taking  each  of  four 

forms  of  a  personality  test  in  three  orders  of  presi  ntati<  n 


[tern 


Obtained  Chi-square  Value 


Test  I- 1 


Test  I-2 


Test  D-i 


I       >    D-2 


I-D"     Alt.b     D-Ic 


I-D      Alt.       D-I 


I-D      Alt.       1)1 


I -I)      Alt.       D-I 


1 

2 
3 
4 

5 

6 

7 
8 

9 
10 

1 1 

12 

13 
14 
15 
16 

17 
18 

iy 
20 

21 
22 
23 
24 
25 
26 

27 
28 

29 
30 

31 
32 

33 
34 
35 
36 


4-4 


4-4 


16  .9 
3-7 


J7-5 


3-7         4-2 


4.0 

3-7 


—        20.8 


6.9  4.4 


10.6 


21.2 


3-7 
5-9       — 


4-7 


6.0 


3-8 


4.2 


—        14.7 
8.9 


3-8  8.5 


5-5 


3-7        7-4 

9-3        5-i 


4.6 
5-2 


5-4 
9-3 
7-3 


7-i 
4.2 


0  •  D 

8.9 


5-5 


3-7 


4.0 


16 . 2 


7-i 


B  Indirect-direct  order  of  presentation. 
b  Alternate  order  of  pres<  ntation. 
0  Direct-indirect  order  of  presentation. 

d  Blank  spaces  represent  non-s.gnificant  Chi-square  values;  all  printed  Chi-square'  values  are  signifi- 
cant at  least  at  the  5%  level  of  confidence. 


orders  of  Tests  D-i   and  D-2  was  as  fol- 
lows: 

No.     5  Finds  it  hard  to  do  his  school 

work  right. 
No.     9  Feels  other  members  of  family 

are  treated  better  than  he. 


No.   10  Finds  school  a   hard   ]>laee-  to 

get  along  in. 
No.   12  Finds  it  hard  to  speak  out  in 

class  recitations. 
No.   13  Feels    that    his    teachers    tic.it 

him  badly  or  unfairly. 
No.    18   Feels  tired  when  he  wakes  up 
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in  the  morning. 
No.  26  Dislikes  school. 
No.  27  People  find  fault  with  him. 

Five  <>f  these  eight  questions  are  con- 
cerned with  school  activities;  and  these 
happen  to  be  the  only  questions  on  school 
activities  in  the  entire  test.  Thus,  the 
problem  children  gave  answers  which 
discriminated    them    significantly    from 


responses  to  the  corresponding  I-i  item; 
il  so  of  the  30  children  answering  the 
I-i  item  negatively  also  answered  the 
1)  1  item  negatively;  and  if  40  of  the  44 
children  answering  the  D-i  question 
positively  also  answered  the  correspond- 
ing I-i  question  positively;  then  the  two- 
by-two  table  set  up  for  determining 
possible  item  identity  from  this  D-i  to 
1-2  item  would  look  like  this: 


Test  I-i 


40 

Positive  on  D-i 

anil 
Positive  on  I-i 

10 
Positive  on  D-i 

and 
Negative  on  I-i 

50 

Total  Positive 

on 

D-i 

4 
Test  D-i                  Negative  on  D-i 

and 
Positive  on  I-i 

20 
Negative  on  D-i 

and 
Negative  on  I-i 

24 

Total  Negative 

on 

D-i 

44 
Total  Positive 
on  I-i 

30 
Total  Negative 
on  I-i 

74 
Total  Positive 

and  Negative 

the  non-problem  children  when  specifi- 
cally questioned  about  their  school  activi- 
ties. 

Problem  6.  Did  item  identity  take 
place  between  the  same  questions  used 
in:  Tests  D-i  and  D-2?  Tests  D-i  and 
I-i?  Tests  D-i  and  1-2? 

Since  Test  D-i  was  the  basic  instru- 
ment used,  and  the  other  three  forms 
were  derived  by  rephrasing  its  questions, 
it  would  be  desirable  to  know  whether 
item  identity  occurred  between  D-i 
questions  and  corresponding  D-2,  I-i  and 
1-2  items.  To  answer  this  problem,  two- 
by-two  tables  were  set  up  showing  how 
the  same  subjects  responded  to  corres- 
ponding items  in  Test  D-i  and  in  each 
of  the  other  tests.  Thus,  if  the  74  non- 
problem  children  taking  the  direct-in- 
direct order  of  the  four  tests  gave  50 
positive  responses  to  a  given  D-i  item 
and  24  negative  responses;  if  these  same 
subjects  gave  44  positive  and  30  negative 


The  specific  point  at  issue  in  this  two- 
by-two  table  is  whether,  out  of  the  24 
children  who  answered  the  D-i  item 
negatively,  the  proportion  who  also  an- 
swered the  I-i  item  negatively  (20/24)  is 
greater  than  the  proportion,  out  of  the 
50  children  who  answered  the  D-i  item 
positively,  who  also  answered  the  I-i 
item  negatively  (10/50).  For  if  it  can  be 
shown  that  a  significantly  higher  pro- 
portion of  the  negative  than  the  positive 
scorers  on  a  D-i  item  also  gave  negative 
answers  to  the  corresponding  I-i  item, 
then  presumably  a  distinct  identity  ex- 
isted between  the  D-i  and  I-i  responses 
to  this  item.  In  other  words,  if  the  hy- 
pothesis of  item  identity  is  correct,  sub- 
jects who  admit  the  possession  of  symp- 
toms like  frequent  crying  on  Test  D-i 
will  have  a  pronounced  tendency  to  re- 
port that  children  who  cry  frequently 
are  not  at  all  queer  (Test  I-i);  that  there 
are  many  such  children  (Test  I-2);  and 
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that  they  are  seldom  or  never  tree  from 
crying  (Test  D-2). 

Statistically,  the  two-by-two  tables 
used  lor  item  identity  analysis  were 
tested  by  the  Chi-square  formula  which 
includes  Yates'  collection: 

[(ad)-(bcj-N/2PN 


(a+b)  (c+d)  (a+c)  (b  +  d) 


When  this  formula  is  employed  with 
the  figures  in  the  two-by-two  table  pre- 
sented above,  Chi-stpiare  is  found  to 
equal  38.4  for  one  degree  of  freedom; 
and  the  null-hypothesis— that  this  is  a 
homogeneous  distribution  and  that  item 
identity  does  not  take  place— may  be 
confidently  rejected. 

In  accordance  with  this  procedure,  the 
36  questions  on  each  of  the  four  tests 
were  subjected  to  an  analysis  to  determine 
whether  item  identity  occurred  from 
Tests  D-i  to  D-2,  D-i  to  I-i,  and  D-i  to 
I-2.  The  results  of  this  analysis  for  the 
non-problem  children's  responses  are  pre- 
sented in  Table  10.  Only  significant  Chi- 
square  values  are  listed  in  the  table.  The 
total  columns  were  obtained  by  adding 
independent  Chi-squares  for  the  three 
order  columns  under  each  test  heading 
and  using  three  instead  of  one  degree  of 
freedom.* 

Several  observations  may  be  made 
from  Table   10: 

1 .  There  was  considerable  item  iden- 
tity between  non-problem  children's  re- 
sponses to  Test  D-i  and  D-2  items.  But 
D-i  to  I-i  and  D-i  to  1-2  identities  were 
far  less  than  the  D-i   to  D-2  identities. 

2.  There  was  decidedly  more  item 
identity  from  D-i   to  1-2  than  from  D-i 

*  Strictly  speaking,  this  is  not  entirely  accu- 
rate, since  the  use  of  Yates'  correction,  accord- 
ing to  Snedecor  (21,  p.  193)  interferes  with  the  ad- 
ditive nature  of  Chi-square.  The  figures  in  the 
total  columns  of  Table  10  may  therefore  be  taken 
only   as  suggestive. 


to  I  1.  Children  who  admitted  the  pos- 
session of  unfavorable  personality  symp- 
toms were  more  willing  to  see  many 
other  children  as  possessing  these  symp- 
toms than  to  call  others  who  possessed 
them  "queer." 

<;.  Mole  epic  slums  showed  item  iden- 
tity than  were  significantly  discriminat- 
ing (according  to  Table  g)  Inn.  ■  n 
problem  and  non-problem  children. 

4.  The    subject    matter    of    the    items 
which  showed  the  most  consistent  id<  n 
tity   from   D-i    to   D-2,    D-i    to    It,    and 
D-i  to  1-2  was  as  follows: 

No.     4  Feels  like  running  awa\   from 

home. 
No.    13  Feels    that    his    teachers    treat 

him  badly  or  unfairly. 
No.  22  Is  afraid  of  high  places. 
No.  26  Dislikes  school. 
No.  32  Finds    it    hard    to    become    a 

leader  in  games  with  other  children. 
No.  36  Is  afraid  of  his  father. 

These  items  covered  the  whole  range 
of  questions  used  in  the  tests.  There 
does  not  seem  to  be  any  consistent  pat- 
ternings  among  them. 

When  an  item  analysis  w"as  done  with 
the  test  results  of  the  problem  subjects. 
Chi-square  values  sIioavii  in  Table  1 1 
were  obtained.  This  time,  however,  the 
three  orders  of  presentation  for  each 
test  were  combined,  since— as  the  analy- 
sis of  variance  of  mean  scores  showed- 
there  were  no  significant  order  differ- 
ences for  the  problem  children.  More- 
over, there  were  too  few  problem  sub- 
jects taking  each  order  t<»  allow  E01 
effective  subdivisional  calculations. 

Examination  of  'Fable  11  reveals  the 
following: 

1.  Item  identity  from  Test  D-i  to  D-2 
was  not  as  prominent  as  it  was  with  the 
non-problem    subjects;    but    D-i     to    I-2 
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Table  io 

Chi-square  values  indicating  a  significant  degree  of  item  identity  from  the  items  on  one 

personality  test  to  the  equivalent  items  on  three  other  personality  tests  taken 
by  221  non-problem  children  in  three  orders  of  presentation 


Obtained  Chi-square 

Value 

hem 

rest  D- 

1  to  1> 

•2 

Test  D- 

1  to  I-I 

Test  D- 

1  to  I- 

2 

I-D» 

Alt.1' 

D-I" 

Tot.d 

I-D      Alt. 

D-I      Tot. 

I-D      Alt. 

D-l 

Tot. 

i. 

4.4      — 

4-0 

9-4 

2. 

3-5 

—        — 

[6.8 

to.  a 

,^- 

[2.8 



6.  a 

21.8 

—        — 

—        — 

S-5 

- — 

— - 

4- 

-4    7 



3-4 

3° -3 

5-4       — 

—        — 

19.4 

12.0 

32.1 

5- 
6. 

12.0 

— 

9.9 

25-1 

8. 

4.0 

151 

— 

10-2 

—         — 

—        — 

9-7 

9.1 

18.9 

u. 

— 

36.9 

37-9 

74.  S 

—         — 

3-7       — 

—         — 

- — 

— 

IO. 

5.6 

6.8 

10.4 

—                  — 

6.7       — 

— 

9-5 

1  1 

_~ 

1  2. 

— 

— 

36.1 

39-4 

4.0 

■ — ' 

13- 

7.0 

18.6 

— 

27 . 2 

6.7       - 

4-9      US 

—         4.2 

- — 

— 

M- 

1 1  .  2 

— 

16.8 

3°-4 

—         5-8 

8.4 

—         — 

— 

— 

*5- 

— 

5.0 

— 

— 

—         — 

—         — 

9.2 

— 

H-3 

10. 

iS. 

— 

11.0 

6-3 

18.0 

—         — 

—         — 

— 

14.  1 

10.5 

5  •  0 
28.9 

53-4 

—         — 

—         — 

18.2        7.3 

3-9 

4-7 

30.2 

10. 

iS.  1 

— 

4.9 

23.0 

20. 

7  ■  5 

10.0 

5-9 

29.4 

21. 

9.4 

22.1 

— 

3i-5 

3-7 

— 

22. 

5-9 

12.5 

13-5 

31-9 

—         — 

10. 1      14.5 

3-7 

.  — 

— 

23- 

29.7 

15.8 

0-5 

52.0 

24- 

— 

11. 8 

15.2 

27.6 

—       15.0 

5.0     20.1 

—         — 

— 

— 

25- 

5-o 

8.6 

— 

14. 1 

—         — 

—         — 

10.9 

— 

13.8 

20. 

27.6 

13.8 

— 

41.8 

9.4     11 . 8 

7-5      28.7 

3.7      11. 6 

— 

16.6 

27- 

— 

— 

9.6 

9-7 

2S. 

6.2     22.8 

— 

29-3 

29. 

3°-5 

20.  7 

36.5 

87.7 

37-9       3-8 

—       43-i 

—         — 

— 

3°- 



14-8 



19. 1 



—         — 

—         — 

— 

— 

3i- 

25.8 

— 

26.6 

52.8 

32. 

J5i 

24.7 

■ — 

41 .0 

4.0       — 

3-5        7-7 

9-5 

— 

10. 1 

33- 

3-3       — 

34- 

6.9 

11 .0 

— 

19-3 

4.0 

3-7 

7.8 

35- 

— 

16.9 

4.0 

20.9 

—         — 

—         — 

4.2 

— 

— 

36. 



16.3 

12.3 

28.8 

4.2 

8.6 

21.9     11. 7 

— 

3S-i 

tt  Indirect-direct  order  of  presentation. 

b  Alternate  order  of  presentation. 

c  Direct-indirect  order  of  presentation. 

d  Totals  are  obtained  by  adding  the  independent  Chi-squares  in  the  three  orders  and  using  three 
degrees  of  freedom. 

e  Blank  spaces  represent  non-significant  Chi-square  values;  all  printed  Chi-square  values  are  signifi- 
cant at  least  at  the  5%  level  of  confidence. 


identity   was   somewhat   greater.   As   be-  tities  made  by  the  non-problem  and  by 

fore,  D-i  to  I-i  identity  was  considerably  the  problem  children. 
less  than  D-i  to  1-2  identity.  3.  The  subject  matter  of  the  questions 

2.  When    Table    11    is    compared    to  showing    the    most    consistent    identity 

Table  10  there  seems  to  be  only  a  moder-  from  Tests  D-i   to  D-2,  I-i,  and  1-2  was 

ate  connection  between  the   item   iden-  as  follows: 


PHRASING    IN    PERSONALITY   QUESTIONNAIRES 


21 


No.   10  Finds  school  a   hard  place  to 

get  along  in. 
No.   12  Kinds  it  hard  to  speak  out  in 

(lass  recitations. 
No.  24  Gets  angry. 
No.  26  Dislikes  school. 

Three  of  these  four  questions  concern 
Table  i  i 

Chi-square  values  indicating  a  significant  degree 

of  item  identity  from  the  items  of  otic  personality 

test  to  the  equivalent  items  on  three  other  per- 

sotlality  tests  taken  by  problem  children 

(Three  orders  of  presentation  combined) 


Obta 

ned  Chi-square 

value 

Item 

Test  D-i 

Test  D-i 

Test  D-i 

to  D-2 

to  I-I 

tO  1-2 

1. 

. * 

— 



2. 

13-8 

— 

— 

3- 

* — 

• — 

— 

4- 

7-8 

— 

19.4 

5- 
6. 

9.4 

— 

IO.  2 

7- 
8. 

— 

3-3 

4.1 

Cl- 

8.4 

— 



io. 

13-9 

8.2 

17.O 

11. 

— 

--  - 



12. 

— 

8.2 

24.7 

13- 

18.7 

— 

— 

14. 

— 

— 



15- 

— 

— 

4-4 

16. 

— 

— 

— 

r7- 
18. 

4.8 

— 

3-7 

19. 
20. 

13.8 

~ 

" 

21. 





22. 

10.  2 

— 

— 

23- 

9.2 

— 

5-9 

24. 

3-3 

5-5 

— 

25- 

19.4 

— 

9-7 

26. 

5-4 

10.  7 

14.6 

27. 

— 

— 

— 

28. 

— 

— 

9.0 

20- 

24.4 

— 

3°- 



— 

— 

3*- 

11. 9 

— 

9.9 

32. 

— 

— 

6-3 

33- 

— 

— 

3-7 

34- 

6.1 

— 

55 

35- 

— 

— 

— 

36. 

4-  1 

— 

— 

*  Dashes  represent  non-significant  Chi-square 
values;  all  printed  Chi-square  values  are  signifi- 
cant at  least  at  the  5%  level  of  confidence. 


school  activities.  This  would  indicate 
that  the  item  diiieientiatimis  «»l  Table 
(),  which  also  were  heavily  loaded  nub 
school  items,  may  have  been  caused  by 
the  problem  children's  reacting  quite 
strongly  to  these  items,  and  admitting 
negative  symptoms  on  them  not  only  on 
the  direct  but  also  the  indirect  ques- 
tions, Their  reactivity  in  this  connection 
may  have  possibly  been  the  result  of 
their  being  already  known  as  problem 
children*  and  therefore  having  little  to 
lose  by  admissions  of  unfavorable  atti- 
tudes towards  school  activities. 

Problem  7.  Did  the  degree  ol  efflbai 
rassment  which  the  subjects  felt  in  an- 
swering  the    test    items   affect    their    re- 
sponses? 

Since  analysis  has  shown  that  certain 
items  on  the  test  forms  discriminate  cl 
significantly  between  non-problem  and 
problem  children,  and  that  item  identity 
occurred  on  some  of  the  questions,  it 
may  be  asked  whether  the  discrimina- 
tions or  item  identities  are  related  to  the 
degree  of  embarrassment  which  the  sub 
jects  experienced  with  the  various  ques- 
tions. To  answer  this  problem,  the  dire<  t 
questions  employed  in  the  experiment 
(that  is,  Test  D-i)  were  resubmitted, 
three  months  after  the  original  admini- 
stration, to  one  hundred  of  the  original 
non-problem  subjects.  This  time,  how- 
ever, the  questions  were  given  anony- 
mously, and  the  children  were  requested 
to  note  whether  they  considered  each 
one  to  be  an  embarrassing  or  a  non- 
embarrassing  one  to  which  to  respond. 
The  results  of  this  supplementary  survey 
of  the  subjects'  attitudes  towards  the 
direct  questions  are  shown  in  Table  12. 

From  Table  12  it  may  be  noted  that. 
in  general,  the  children  found  those 
questions  embarrassing  which  asked 
whether  other  people  were  critical  ol 
them,  whether  they  found  it  hard  to  get 
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Table  12 


Percentage  of  one  hundred  non-problem  school  children  finding 
personality  test  items  embarrassing 


Item 


Subject  Matter  of  Item 


Percentage  of 

children 

finding  item 

embarrassing 


1. 

j. 

3- 

4- 

5- 

6. 

S. 

9- 

10. 


Would  rather  play  by  self  than  with  other  children 

Finds  it  hard  to  make  friends  with  other  children 

Cries 

Feels  like  running  away  from  home 

Finds  it  hard  to  do  school  work  right 

Is  afraid  of  the  dark 

Feels  ill  or  weak 

Is  afraid  of  being  followed  when  walking  on  the  street 

Fe<  1>  other  members  of  family  are  treated  better  than  he 

Finds  school  a  hard  place  to  get  along  in 


ii.  Other  children  refuse  to  play  with  him 

1  2.   Finds  it  hard  to  speak  out  in  class  recitations 

13.  Feels  that  his  teachers  treat  him  badly  or  unfairly 

14.  His  feelings  are  easily  hurt 

15.  Feels  that  nobody  really  loves  him 


16. 

17- 
18. 
19. 
20. 


Feels  tired  during  the  daytime 

Finds  home  a  hard  place  to  get  along  in 

Feels  tired  when  he  wakes  up  in  the  morning 

Is  afraid  during  a  thunder-storm 

Has  headaches 


21.  Is  bothered  with  pains  in  some  part  of  his  body 

22.  Is  afraid  of  high  places 

23.  Day-dreams 

24.  Gets  angry 

25.  Has  the  habit  of  twitching  his  neck  or  face 


26. 

27- 

28. 
29. 

3°- 


Dislikes  school 

People  find  fault  with  him 

Parents  blame  or  find  fault  with  him 

Bites  his  nails 

Is  afraid  of  fire 


31.  Has  horrible  dreams  while  sleeping 

32.  Finds  it  hard  to  become  the  leader  in  games  with  other  children 
^^.  Becomes  gloomy  or  sad 

34.  Becomes  excited 

35.  Stutters  or  stammers  when  talking 

36.  Is  afraid  of  his  father 


3* 
14 
25** 
29** 
19** 


10 
11 
11 

13 
11 

14 

8 

13 

26** 

12 


15 
4* 
7 
1* 


i" 

12 
8 


20' 
19* 
17* 


22' 
8 
10 
11 
13 


'  The  nine  questions  which  were  considered  embarrassing  by  the  fewest  children. 
**  The  nine  questions  which  were  considered  embarrassing  by  the  most  children. 


along  with  others,  and  whether  they  were 
sensitive  to  criticism.  Thev  found  rela- 

J 

tively  non-embarrassing  the  questions 
which  asked  if  they  had  physical  ail- 
ments or  were  afraid  of  things  like  fire  or 
high  places.  There  are  not  many  sur- 
prises in  the  tabulated  results. 

When  the  facts  of  Table  1 2  are  viewed 


against  those  of  Tables  9  and  10,  no 
clear-cut  results  can  be  discerned.  Thus, 
of  the  items  in  Table  9  which  signifi- 
cantly differentiated  non-problem  from 
problem  children,  three  of  those  of  Test 
I-i  fell  in  the  most-embarrassing,  three 
in  the  least-embarrassing  categories,  and 
the  other  six  in-between.  Of  the  four  I-2 
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discriminating  items,  one  was  least-em- 
barrassing, and  three  in-between.  Ol  the 
seventeen  D-2  discriminating  items,  one 
was  in  the  least-embarrassing  area,  five 
in  the  most  embarrassing,  and  eleven 
in-between.  Of  the  ten  discriminating 
D-i  items,  two  fell  in  the  least-embar- 
rassing category,  three  in  the  most-em- 
barrassing and  five  in-between.  There  is 
consequently  little  to  indicate  that  the 
degree  of  embarrassment  which  the  chil- 
dren said  they  felt  affected  the  discrim- 
inative power  of  the  test  items. 

Comparing  the  Table  12  values  to 
those  of  the  significant  item  identities 
listed  in  Table  10,  results  are  similarly 
inconclusive.  Thus,  of  the  twenty-seven 
item  identities  from  Test  D-i  to  D-2, 
eight  fell  in  the  most-embarrassing  cate- 
gory, six  in  the  least  embarrassing,  and 
thirteen  in-between.  Of  the  eight  item 
identities  from  Test  D-i  to  I-i,  one  was 
most-embarrassing,  three  least-embarrass- 
ing and  four  in-between.  Of  the  thirteen 
identities  from  Test  D-i  to  1-2,  three 
fell  in  the  most-embarrassing,  three  in 
the  least-embarrassing,  and  seven  in  the 
in-between  categories.  It  is  evident,  there- 
fore, that  according  to  the  children's 
own  attitudes  towards  degree  of  embar- 
rassment experienced  in  answering  the 
direct  test  items,  there  was  no  noticeable 
degree  of  correspondence  between  em- 
barrassment or  non-embarrassment  on 
the  one  hand  and  item  identity  on  the 
other. 

Problem  8.  How  were  the  responses 
of  the  normal  children  affected  by  the 
subject  matter  of  the  items  of  the  four 
tests? 

To  discover  whether  the  subject  matter 
of  the  items  of  the  four  tests  affected 
the  children's  responses,  the  thirty-six 
questions  of  each  form  were  divided  into 
six  categories,  according  to  their  con- 
tent. These  categories  were:  (1)  questions 


relating  to  home  adjustment;  (2)  to 
social  adjustment;  (3)  to  school  adjust- 
ment; (4)  to  fears;  (5)  to  sensitivity  and 
excitability;  and  (6)  to  psychosomatit 
symptoms.  Listed  in  Table  13  are  the 
six  categories,  with  the  percentages  ol 
non-problem  children  responding  nega- 
tively or  unfavorably  to  each  item.  The 
problem  subjects  are  not  included  since 
the  small  number  of  cases  make  theii 
percentage  figures  meaningless. 

Table  13  shows  distinct  differences 
among  the  average  percentages  of  sub 
jects  responding  negatively  to  the  qu<  s 
tions  in  each  of  the  six  subject  cate- 
gories. When  tested  with  analysis  of 
variance,  these  differences  are  significant 
for  Tests  I-i,  1-2,  and  D-i,  but  not  Eoi 
D-2.  None  of  the  order  differences  are 
significant. 

If  the  percentages  in  Table  13  are 
averaged  by  orders  (since  the  order  dif- 
ferences are  not  significantly  different) 
the  summary  information  shown  in 
Table  13a  is  obtained.  From  Tables  13 
and  13a  it  may  be  seen  that: 

1.  On  all  four  tests  the  subjects  were 
least  willing  to  respond  negatively  to  the 
home  adjustment  and  social  adjustment 
questions  and  most  willing  to  respond 
unfavorably  to  the  psychosomatic  and 
sensitivity  and  excitability  questions. 

2.  There  was  some  degree  of  corre- 
spondence between  the  manner  in  which 
the  children  responded  to  the  six  cate- 
gories of  questions  on  all  four  tests,  but 
there  was  also  some  degree  of  discrep- 
ancy. 

Additional  information  may  lie  ob- 
tained by  noting  the  nine  most-checked 
and  the  nine  least-checked  items  for  ea<  h 
of  the  four  tests.  This  information  is 
shown  in  Table  14.  From  Tabic  1  1  it 
may  be  seen  that  on  several  of  the  ques- 
tions—especially those  concerned  with 
the  children's  running  awaj  from  home. 
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Table  13 

Percentages  of  221  non-problem  children  giving  unfavorable  responses  to  the  items  of  four 
personality  tests  taken  in  three  orders  of  presentation 


Percentage  of  Chilli 

ren  G 

iving  Unfavorabl 

e  Res 

ponse 

Subject  Matter  of  Item 

Test  I- 1 

Test  1-2 

Test  D-i 

Test  D- 

2 

1-Dft 

Ml1' 

D-I° 

I-D 

Alt. 

D-I 

I-D 

Ml 

D-I 

I-D 

Alt. 

D-I 

Home  Adjustment: 

Feels  like  running  from  home 

17 

17 

1 .' 

16 

0 

4 

7 

3 

3 

6 

3 

7 

Feels  family  treated  better 

34 

43 

39 

in 

13 

15 

1 

7 

3 

1 

7 

4 

I'inds  home  a  hard  place 

17 

10 

21 

3 

7 

7 

3 

4 

1 

1 

5 

4 

Feels  parents  blame  him 

3° 

.•0 

29 

It) 

21 

18 

17 

13 

8 

17 

25 

20 

Is  afraid  of  father 

39 

30 

30 

21 

34 

28 

21 

14 

8 

16 

12 

14 

AVERAGE 

29 

2S 

26 

15 

17 

14 

10 

8 

5 

s 

IO 

IO 

Social  Adjustment; 

Plays  by  himself 

23 

39 

23 

4 

7 

4 

6 

8 

8 

7 

14 

9 

Finds  it  hard  to  make  friends 

26 

31 

28 

7 

19 

11 

3 

8 

5 

3 

9 

5 

Other  boys  won't  play 

23 

39 

27 

17 

1  2 

9 

0 

4 

4 

0 

5 

1 

Feels  nobody  loves  him 

14 

13 

14 

8 

10 

1  2 

1 1 

14 

1 

6 

3 

8 

Day-dreams  often 

33 

35 

32 

•|i 

35 

36 

13 

14 

18 

13 

14 

5 

Hard  to  become  a  leader 

5« 

44 

52 

45 

41 

32 

21 

21 

35 

19 

26 

13 

AVERAGE 

30 

34 

30 

21 

21 

18 

8 

12 

12 

8 

I  2 

10 

School  Adjustment: 

Hard  to  do  school  work 

54 

34 

50 

36 

36 

28 

4 

10 

8 

6 

7 

3 

Finds  school  a  tiard  place 

33 

25 

18 

36 

36 

32 

1 1 

9 

7 

3 

5 

5 

Hard  to  speak  in  class 

S2 

46 

53 

29 

23 

36 

1 1 

3 

30 

16 

26 

31 

Teachers  treat  him  badly 

33 

36 

36 

41 

50 

28 

13 

12 

12 

,7 

9 

5 

Dislikes  school 

41 

34 

35 

60 
41 

42 
37 

48 
34 

21 

14 

12 

11 

7 

11 

AVERAGE 

43 

35 

39 

I  2 

10 

14 

9 

11 

11 

Fears: 

Is  afraid  of  the  dark 

33 

26 

25 

39 

38 

31 

6 

10 

8 

8 

7 

5 

Afraid  of  being  followed 

26 

27 

19 

30 

27 

32 

7 

9 

8 

IQ 

9 

II 

Afraid  during  thunder-storm 

36 

40 

26 

30 

22 

26 

6 

12 

8 

4 

7 

1 1 

Afraid  of  high  places 

53 

46 

60 

50 

58 

46 

7 

17 

18 

13 

19 

16 

Afraid  of  fire 

80 

61 

54 

40 

39 

35 

6 

12 

3 

IO 

18 

1 1 

Has  horrible  dreams 

3° 

35 

35 

3<> 

29 

27 

1 1 

5 

3 

8 

5 

4 

AVERAGE 

43 

39 

37 

38 

36 

38 

7 

1 1 

8 

10 

1 1 

10 

Sensitivity  &  Excitability: 

Cries  easily 

37 

30 

48 

56 

25 

46 

10 

12 

9 

11 

12 

IO 

Feelings  easily  hurt 

36 

39 

50 

41 

36 

35 

10 

15 

25 

8 

10 

15 

Gets  angry 

30 

19 

38 

80 

74 

59 

24 

23 

30 

23 

26 

26 

People  rind  fault  with  him 

41 

30 

23 

2  1 

17 

20 

4 

7 

1  2 

8 

9 

16 

Feels  gloomy  or  sad 

40 

46 

18 

30 

27 

19 

3 

3 

7 

3 

19 

5 

Becomes  excited 

47 

39 

44 

73 

54 

42 

29 

27 

27 

1 1 

5 

18 

AVERAGE 

42 

34 

37 

50 

39 

36 

13 

IS 

18 

11 

13 

14 

Psychosomatic  Symptoms: 

Feels  ill  or  weak 

64 

46 

50 

37 

3° 

18 

4 

3 

3 

6 

4 

16 

Tired  during  the  daytime 

54 

50 

46 

23 

27 

18 

6 

8 

7 

4 

8 

4 

Tired  when  waking 

80 

82 

72 

6b 

68 

59 

37 

39 

39 

26 

26 

9 

Has  headaches 

66 

70 

41 

52 

27 

35 

22 

17 

13 

8 

10 

9 

Pains  in  body 

71 

52 

68 

33 

23 

19 

13 

5 

15 

7 

8 

4 

Twitches  neck  or  face 

27 

34 

36 

7 

18 

9 

1 

17 

5 

4 

8 

8 

Bites  nails 

54 

52 

50 

78 

60 

64 

33 

29 

38 

23 

15 

32 

Stutters  or  stammers 

67 

57 

55 

27 

29 

22 

1 

7 

15 

3 

7 

8 

AVERAGE 

60 

55 

52 

40 

35 

30 

14 

16 

17 

13 

1 1 

1 1 

a — Indirect-Direct  Order 

b- 

-Alternate  Order 

c — Direct-I 

ndirtct  Order. 

finding  it  hard  to  make  friends,  tired- 
ness when  waking  in  the  morning,  and 
biting  their  nails— there  was  a  definite 
tendency  for  the  subjects  to  give  the 
same  kind  of  response  to  all  four  tests. 
But  on  a  few  of  the  questions— specifi- 
cally, those  concerned  with  the  subjects' 
feeling  their  parents  blamed  them,  being 


afraid  of  being  followed,  feeling  gloomy 
or  sad,  and  feeling  ill  or  weak— they 
freely  gave  unfavorable  responses  to  one 
form  of  the  test  but  rarely  gave  them 
to  another  form. 

Table  14  confirms  the  information  of 
Tables  13  and  13a:  to  wit,  that  non- 
problem  subjects,  on  all  four  tests,  ad- 
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1  ABLE   13a 

Average  percentages  of  non-problem  children  giving  unfavorable  responses  to  six  categories 
of  questions  on  four  personality  tests  (three  orders  of  presentation  combined  I 


Average  Percentage  Giving 

Unfavorable 

Response 

Category  ot  Questions 

Test  D-i 

Test  D-2 

Test  1-2 

1.   -!    1-2 

Home  Adjustment 

7-7 

9-3 

26.8 

15-3 

Fears 

8.7 

10.3 

39-6 

37-3 

Social  Adjustment 

10.7 

10. 0 

31-3 

20.0 

School  Adjustment 

12.0 

10.3 

39° 

37-3 

Sensitivity  and  Excitability 

15-3 

12.7 

37-7 

41 .7 

Psychosomatic  Symptoms 

15-7 

11. 7 

55-7 

35-0 

mitted  with  relative  rarity  that  they 
were  unadjusted  to  their  home  and  that 
they  were  socially  inadequate;  but  more 
freely  admitted  that  they  were  sensitive 
and  excitable  and  that  they  possessed 
psychosomatic  symptoms.  This  pattern 
of  response  may  be  attributable  to  the 
fact  that  (1)  the  children  really  did  possess 
fewer  unfavorable  home  and  social  symp- 
toms than  sensitivity  and  psychosomatic 
ones;  or  (2)  they  felt  less  self-criticism  in 
admitting  the  latter  than  the  former 
kind  of  symptoms.  To  check  this  second 
possibility,  the  information  in  Table  12, 
showing  the  degree  of  embarrassment 
felt  by  non-problem  children  in  answer- 
ing the  direct  items,  may  be  compared 
with  the  data  of  Table  14.  It  is  then 
found  that  of  the  nine  questions  con- 
sidered most-embarrassing  by  the  chil- 
dren, three  were  among  the  least-checked 
items  in  one  or  more  of  the  tests,  four 
were  among  the  most-checked,  one  was 
least-checked  in  two  tests  and  most- 
checked  in  one,  and  one  was  neither 
among  the  most-  or  least-checked  items. 
Of  the  nine  items  considered  least-em- 
barrassing by  the  children,  five  were 
among  the  most-checked  items  on  one  or 
more  tests,  and  four  were  among  the 
least-checked.  Apparently,  there  was  not 
a  high  degree  of  correspondence  between 
the  children's  finding  specific  items  em- 
barrassing, on  the  one  hand,  and  giving 


unfavorable  answers  to  them  on  the 
other  hand.  The  observed  dilferences 
among  the  six  categories  of  questions 
may  therefore  be  attributable  to  actual 
differences  in  the  subjects'  possession  of 
the  various  traits  rather  than  the  result 
of  their  responding  more  unfavorabh  to 
the  less  self-critical  items.  Certainly,  it 
is  easy  to  believe,  as  the  data  of  Table  14 
indicate,  that  non-problem  children  less 
often  feel  like  running  away  from  home 
and  less  often  find  it  hard  to  make- 
friends  with  other  children  than  they 
feel  tired  when  waking  in  the  morning 
and  than  they  bite  their  nails. 

Problem  p.  What  was  the  effect  of 
the  children's  answering  certain  direct 
and  indirect  questions  out  of  context  as 
against  answering  identical  questions  as 
part  of  the  regular  test  battery? 

The  test  booklet  used  in  the  investiga- 
tion contained  three  sample  questions  <>n 
their  covers.  The  sample  I)-i  question 
was:  "I  get  angry  VERY  OF  1  I  N 
PRETTY  OFTEN  SELDOM  NEVER." 
The  sample  I-i  quesion  was:  "Children 
who  often  day-dream  are  VERY  QUEER 
PRETTY  QUEFR  A  III  II  1 
QUEFR  NOT  AT  ALL  QUI  I  K 
The  sample  1-2  question  was:  "VERY 
MANY  PRETTY  MANY  FEW  No 
children  are  afraid  o\  high  places."  No 
sample  D-2  question  was  employed,  since 
its  form  was  similar  to  that  ol   the   D-i 
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Table  14 

The  nine  most  frequently  checked  and  nine  least  frequently  checked  items  on  four 
personality  tests  taken  by  2-1  non-problem  children  (orders combined) 


Subject  Matter  of  Item 


Most-Checked 

Test 

I-I         1-2       D-I      D-2 


Least-Checked 

Test 

I-i       1-2      D-i     D-2 


Home  Adjustment: 

Feels  like  running  away  from  home 
Feels  family  treated  better  than  he  is 
Finds  home  a  hard  place 
Feels  parents  often  blame  him 
Is  afraid  of  his  father 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Social  Adjustment : 
Plays  by  himself 
Finds  it  hard  to  make  friends 
Other  boys  refuse  to  play  with  him 
Feels  that  nobody  loves  him 
Day  dreams  often 
Finds  it  hard  to  become  a  leader 


School  Adjustment : 

Finds  it  hard  to  do  school  work 
Finds  school  a  hard  place 
Finds  it  hard  to  speak  in  class 
Feels  teachers  treat  him  badly 
Dislikes  school 


x 

X 


X 
X 


Fears: 

Is  afraid  of  the  dark 

Is  afraid  of  being  followed  when  walking 

Is  afraid  during  a  thunder-storm 

Is  afraid  of  high  places 

Is  afraid  of  fire 

Has  horrible  dreams 


x 
x 


X 
X 


X 
X 
X 


Sensitivity  and  Excitability: 
Cries  easily 

Feelings  are  easily  hurt 
Often  gets  angry 
Feels  people  find  fault  with  him 
Feels  gloomy  or  sad 
Becomes  excited 


x 

X 


X 

X  X 


tychosomatic  Symptoms : 

Feels  ill  or  weak 

X 

Feels  tired  during  the  daytime 

Is  tired  when  waking  in  the  morning 

X 

X 

X 

Has  headaches 

X 

X 

X 

Has  pains  in  his  body 

X 

Twitches  his  neck  or  face 

Bites  his  nails 

X 

X 

X 

Stammers  or  stutters 

X 

X 

X  X 


X  X 


questions.  The  children  were  not  spe- 
cifically requested  to  answer  these  sample 
questions,  but  101  of  the  non-problem 
children  voluntarily  did  so,  presumably 
as  aids  to  themselves.  Accordingly,  since 
the  sample  questions  were  also  contained 


in  the  regular  context  of  the  tests,  the 
two  responses  of  these  101  subjects  to 
each  of  the  three  sample  questions  were 
compared  to  see  if  the  contextual  one 
was  answered  in  (1)  a  more  favorable, 
(2)  a  less  favorable,  or  (3)  an  unchanged 
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manner  from  the  non-contextual  re- 
sponses. The  results  of  the  comparisons 
are  shown  in   Table  15. 

From  Tabic  15  the  following  points 
may  be  noted: 

i.  Almost  one-fourth  of  the  children 
answering  the  sample,  non-contextual 
direct  question  changed  their  contextual 
response  to  a  more  favorable  one.  At  the 
same  time,  only  4%  changed  their  re- 
sponse less  favorably.  If  it  is  assumed 
that  the  non-contcxtnal  answers  were  the 
more  truthful  ones— since  they  were  pre- 
sumably given  when  the  children  were 
caught  off  their  guard,  and  given  to  ques- 
tions on  which  they  felt  they  would  not 
be  marked— there  is  some  evidence  that 
sophistication  occurred  as  a  result  of  the 
test  situation.  And  since  the  particular 
direct  questioned  involved  here— that 
asking  whether   the  children   often   got 


angry— was    one     which,     according     to 
Tables    12    and    14,    the   subjects    freely 
responded   to  unfavorably,   and   accord 
ing  to    Table    13  did   not  deem   partii 
ularly  embarrassing,  there  is  souk    rea 
son  to  believe  that  the  test  sophistication 
shown  here  also  would  prevail  for  most 
other  direct  questions. 

2.  The  direction  of  the  changes  made 
in  the  1-2  sample  question  were  similar 
to  those  of  the  D-i  question,  but  not  so 
pronounced  as  to  evidence  of  test  sophis- 
tication. Some  18%  of  the  children 
changed  their  1-2  response  in  a  more 
favorable  direction;  but  10%  also 
changed  it  in  a  less  favorable  manner. 
There  seems  to  be  nothing  conclusive 
about  these  changes. 

3.  The  Test  I-i  sample  question  pro 
duced   results  opposite   to   those   of   the 
other    two    questions.    This    time,    the 


Table  15 

Changes  made  by  101  non-problem  children  in  answering  certain  personality  test 
questions  first  out  of  context  and  then  as  part  of  a  regular  test  battery 


Number  and  percentage  of  children  making  change 


Direction  of  Change 
in  Response 


Indirect- 
Direct 
Order 

N         % 


Alternate 
Order 


N 


% 


Direct- 
Indirect 
Order 

N         % 


Combined 
Orders 


N 


% 


D-i  Question  (gets  angry) 

More  favorable 
Less  favorable 
Unchanged 

Total 

I-i  Question  (day-dreams  often) 

More  favorable 
Less  favorable 
Unchanged 

Total 

1-2  Question  (js  afraid  of  high  places) 

More  favorable 
Less  favorable 
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children  (hanged  their  answers  to  more 
favorable  responses  17%  of  the  time  and 
to  less  favorable  ones  26%  of  the  time. 
All  told,  no  less  than  43%  of  the  sub- 
jects changed  their  contextual  response 
to  this  item.  Several  possible  explana- 
tions lor  this  unusual  response  pattern 
present  themselves:  (1)  the  children  never 
quite  figured  out  what  were  the  favor- 
able answers  to  this  question,  and  were 
therefore  as  willing  to  change  their 
answers  one  way  as  the  other;  (2)  what 
the  scoring  system  assumed  to  be  "favor- 
able" answers  to  this  test  did  not  prove 
to  be  so  empirically;  (3)  the  subject 
matter  of  the  question— that  asking 
about  the  children's  day-dreams— was 
especially  conducive  to  unstable  re- 
sponses. Since  none  of  these  explana- 
tions, individually  considered,  seems 
conclusive,  it  is  possible  that  a  combina- 
tion of  them  accounted  for  the  observed 
changes. 

Because  of  the  differences  in  subject 
matter  of  the  three  sample  questions,  it 
is  not  possible  directly  to  compare  the 
changes  made  on  one  to  those  made  on 
the  other  two.  However,  since  the  three 
are  fairly  equivalent  in  the  sense  that 
none  of  them  was  found  to  be  very  em- 
barrassing by  the  children  (Table  12) 
and  all  tended  to  be  among  the  most- 
checked  responses  (Table  14),  there  are 
some  indications  here  that,  given  the 
scoring  procedures  used  in  this  investi- 
gation, the  D-i  question  was  somewhat 
transparent  to  the  subjects,  the  1-2  ques- 
tion less  transparent,  and  the  I-i  ques- 
tion non-transparent.  This  agrees  with 
the  item  identity  findings,  which  showed 
that  the  1-2  questions  were  more  closely 
related  to  the  D-i  items  than  were  the 
I-i  questions. 

RESUME   OF  THE   FINDINGS 

The  major  findings  of  the  experiment, 


classified  according  to  the  main  variables 
involved    in   it,   are   briefly   listed   below. 

Tests 

1.  The  mean  scores  of  the  direct  tests 
(D-i  and  D-2)  do  not  differ  significantly. 

2.  The  average  scores  of  the  indirect 
tests  (I-i  and  I-2)  ate  significantly  differ- 
ent. 

3.  The  mean  scores  of  the  direct 
tests  (D-i  and  D-2)  differ  significantly 
from  those  of  both  indirect  tests  (I-i 
and    I-2). 

4.  The  observed  mean  score  differ- 
ences are  not  attributable  to  lack  of  in- 
ternal consistency  or  lack  of  item  stabil- 
ity of  the  tests. 

5.  The  intercorrelations  between  Tests 
D-i    and   D-2   are   reasonably   high;    be 
tween  I-i  and  1-2,  fairly  low;  and  between 
the    direct    and    indirect    tests,    not    too 
high. 

6.  Many  item  identities  exist  between 
Test  D-i  and  D-2;  a  good  many  between 
D-i  and  I-2;  but  relatively  few  between 
D-i  and  I-i. 

7.  The  subjects  give  significantly  more 
unfavorable  responses  to  certain  kinds 
of  questions— e.g.,  psychosomatic  ones— 
than  they  do  to  other  kinds— e.g.,  home 
adjustment   or  social   adjustment   items. 

8.  Subjects  taking  specified  test  items 
as  part  of  the  regular  test  battery  respond 
differenly  than  when  taking  identical 
items  out  of  test  context. 

9.  The  degree  of  embarrassment  the 
subjects  say  they  experience  in  answering 
the  direct  questions  does  not  affect  the 
power  of  the  items  to  discriminate  be- 
tween the  two  groups  or  the  item  identi- 
ties of  the  questions. 

Groups 

10.  All  the  obtained  group  differences 
are  to  be  cautiously  viewed,  since  the 
problem  subjects  are  few  in  number  and 
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art'    probably    non-homogeneously    sub- 
g]  ouped. 

11.  Significant  differences  between 
problem  and  non-problem  subjects' 
average  scores  are  evident  only  for  Test 
D-2,   and   not   lor  Tests   I)-i,    I-i,   or   1-2. 

12.  Few  items  discriminate  significant- 
ly between  problem  and  non-problem 
groups  on  any  of  the  four  tests,  though 
the  direct  tests  contain  more  item  dis- 
criminations than  do  the  indirect.  Dif- 
ferent specific  items  prove  discriminating 
on  the  direct  and  the  indirect  tests. 

13.  The  items  which  discriminate 
significantly  between  problem  and  non- 
problem  groups  are  largely  concerned 
with  school  activities. 

Orders  of  Presentation 

14.  The  order  of  presentation  of  the 


direct  and  the  indirect  test  questions  has 
a  signifu  ant  effe<  I   upon: 

a.  Average  test  scores. 

b.  Test  score  intercorrelations. 

c.  Item      discriminations      between 

problem      and      non-problem 
groups. 

d.  Biserial      correlations      between 

total  test  and  item  s(  ores. 

e.  Percentages  of  subjects  respond- 

ing   unfavorably    to    differenl 
kinds  of  questions. 

Interactions 

15.  None  of  the  first-order  01  second- 
order  interactions  among  tests,  groups. 
or  orders  of  presentation  are  statisticallv 
significant. 


Chapter  IV 
Discussion 


Since  statistically  significant  differ- 
ences were  found  Eor  tests,  orders  ol 
presentation,  and  groups  used  in  the 
experiment,  it   may  now  be  asked  why 

they  on  nil  nl  and  what  they  mean. 

INTERPRETATION  OF  THE  TEST 
DIFFERENCES 

The  indirect  tests  employed  in  the 
i  xperiment  differed  significantly  from 
each  other  as  well  as  from  the  direct 
tests;  and  these  differences  were  main- 
tained for  botli  problem  and  non-prob- 
lem subjects.  They  may  be  attributable 
to  differences  between  the  indirect  and 
direct  tests  because  of  (1)  format  or  (2) 
substance. 

It  seems  unlikely  that  the  test  differ- 
ences were  the  result  of  format  differ- 
ences because:  (1)  deliberate  change  in 
phrasing,  but  avoidance  of  change  in 
meaning,  did  not  bring  about  signifi- 
cant differences  between  Tests  D-i  and 
D-2.  (2)  The  formats  of  Tests  I-i  and 
1-2  differed  from  each  other;  yet,  both 
I-i  and  1-2  mean  scores  were  signifi- 
cantly different  from  D-i  and  D-2  scores. 
(3)  Changing  the  order  of  presentation 
of  the  tests,  which  somewhat  changed 
their  appearance  or  format,  had  no 
significant  effect  on  the  distinct  differ- 
ences  found  between  the  direct  and  in- 
direct test  forms. 

Since  there  is  little  reason  to  believe 
that  the  format  of  the  tests  was  re- 
sponsible for  the  observed  mean  score 
differences,  these  would  seem  to  be  at- 
tributable to  differences  in  test  sub- 
stance. 

INTERPRETATION    OF   THE   GROUP 
DIFFERENCES 

It   was   found,   by   statistical    analysis, 


that  the  problem  children  reacted  differ- 
ently than  the  non-problem  subjects  to 
rest  D-2;  but  that  there  were  no  signifi- 
cant differences  between  the  mean 
scores  of  the  two  groups  on  Tests  D-i, 
I-i,  and  1-2.  The  most  plausible  reasons 
for  these  diverse  group  differences  seem 
to  be  as  follows: 

1.  Tests  D-2  and  I-i,  which  most  suc- 
cessfully  differentiated    the    two   groups 
(the  latter  displaying  differences  which 
were   almost   significant),   seem   to   have 
been   the  tests  with  which   the  subjects 
had    the    greatest    difficult    determining 
the  "good"  or  "correct"  answers.   Some 
of    the    D-2    questions   were    awkwardly 
worded    and    hence    liable    to    misinter- 
pretation;   while    few    of    the    I-i    ques- 
tions—as the  item  identity  data  revealed 
—could  be  readily  identified  by  the  sub- 
jects  with    their   D-i    analogues.    Conse- 
quently, the  problem  children,  who  were 
found   to   have   lower   intelligence   quo- 
tients   than    the    non-problem    subjects, 
may  have  answered  the  D-2  and  the  I-i 
items  less  sophisticatedly  than  did  the  lat- 
ter  group:    thereby   producing   the   ob- 
served differences. 

2.  The  group  differences  could  have 
been  caused  by  lack  of  homogeneity 
among  the  subgroups  of  problem  chil- 
dren used  in  the  experiment.  These 
problem  subjects,  not  being  the  most 
acute  and  segregable  kind  of  cases,  neces- 
sarily consisted  of  a  heterogeneous  popu- 
lation which  probably  included  several 
clinical  subgroups.  From  such  a  hetero- 
geneous grouping,  a  sample  as  small 
as  that  employed  in  the  experiment 
could  easily  have  varied  significantly 
from  one  order  of  presentation  to  an- 
other, thus  causing  some  inconsistencies 
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in  the  findings.  Observation  of  Tabic  1 
(p.  7)  tends  to  bear  out  this  possibility 
by  showing  that  while  the  order  of  pre- 
sentation scores  for  the  non-problem 
children  ran  in  a  consistent  direction 
for  all  tests,  those  for  the  problem  chil- 
dren were  either  inconsistent  or  dispro- 
portional  from  order  to  order.  That  is 
why  sophistication  effects  found  for  the 
non-problem  group  were  not  also  found 
for  the  problem  subjects.  The  internal 
consistency  coefficients  (Table  5,  p.  12) 
also  show  evidence  of  inconsistency  in 
the  problem  children's  responses,  and 
this  inconsistency  again  could  have  been 
caused  by  differences  writhin  the  sub- 
groups of  problem  subjects.  Since, 
throughout  the  entire  experiment,  the 
non-problem  children,  whose  sub-groups 
always  included  approximately  seventy 
cases,  showed  fairly  consistent  findings, 
and  the  problem  children,  whose  sub- 
groups included  approximately  thirteen 
cases,  often  showed  inconsistent  and  con- 
tradictory findings,  there  must  be  a 
strong  suspicion  that  the  obtained  differ- 
ences between  the  mean  scores  of  the 
two  groups  resulted  from  unusual  hetero- 
geneity in  the  makeup  of  the  problem 
subgroups.  Consequently,  not  too  much 
emphasis  may  be  placed  on  the  observed 
group  differences  until  they  are  verified 
by  further  experimentation.  The  ob- 
tained test  and  order  of  presentation 
differences,  on  the  other  hand,  stand  up 
much  better  under  statistical  analysis. 

INTERPRETATION    OF    THE    ORDER    OF 
PRESENTATION    DIFFERENCES 

The  obtained  mean  score  differences 
on  the  three  orders  of  presentation  used 
in  the  experiment  seem  to  be  clearly  ex- 
plicable iii  terms  of  test  sophistication. 
Bolstering  the  argument  that  one  pre- 
sentation order  enabled  the  subjects  t<> 
become  more  sophisticated  on   the  tests 


than  did  the  other  orders  are  the  follow- 
ing points: 

1.  The  hypothesis  of  test  sophistica- 
tion would  lead  to  the  expectation  that 
different  orders  of  presentation  would 
produce  greater  differences  on  the  in- 
direct than  on  the  direct  tests,  since  the 
latters'  questions  were  so  openly  stated 
as  to  reveal  their  intent  no  matter  what 
presentation  order  was  employed.  This 
was  precisely  what  occurred,  with  ordei 
differences  being  significant  only  on 
Tests   I-i    and   1-2,   and   not  on   D-i    or 

D-2. 

2.  If  sophistication  effects  existed,  the) 
should  have  been  revealed  by  the  sub- 
jects' receiving  higher  (or  less  favorable) 
scores  on  the  indirect  tests  when  they 
answered  all  the  indirect  questions  be- 
fore the  directly  stated  ones,  than  when 
they  first  answered  the  direct  and  then 
the  indirect  questions.  This  is  exactly 
what  the  experimental  data  reveals  them 
to  have  done. 

3.  If  the  sophistication  hypothesis 
were  tenable,  it  would  be  expected  that 
subjects  taking  the  alternate  order  of 
presentation  would  tend  to  receive  mean 
test  scores  in-between  those  taking  the 
indirect-direct  and  direct-indirect  orders. 
This,  again,  was  revealed  by  the  data.' 

4.  The  sophistication  hypothesis 
would  leave  the  probability  of  some 
degree  of  sophistication  taking  place  on 
the  direct  tests,  since  subjects  answering 
these  tests  last  (that  is,  taking  the  indi- 
rect-direct order)  might  be  made  m>um 
what  test-wise.  The  obtained  scores 
show  that  some  (though  a  statistically 
non-significant)  degree  of  sophistication 
did  occurr  on  the  direct  tests,  and  that 
it  was  precisely  in  the  expected  direc- 
tion. 

On  all  counts,  therefore,  the  h\  pothesis 
set  ins  to  be  upheld;  and  it  would  defi- 
nitely  appear   that   the  observed   differ- 
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ences  produced  by  the  orders  of  pres-  vious  implications  which  go  with  it,  may 
entation  were  attributable  to  test  sophis-  be  said  to  be  the  most  important  dis- 
i  it  ai  ion.      This     finding,     and     the    ob-      covery  of  the  investigation. 


Chapter  V 
Summary  and  Conclusions 


The  purpose  of  the  present  experi- 
ment was  to  study  the  effects  of 
using  direct  and  indirect  phrasing  in 
personality  questionnaires.  The  specific 
hypothesis  to  be  tested  was  that  if  conven- 
tional personality  test  schedules  employ- 
ing direct  questions,  as  well  as  indirect 
test  forms  derived  from  the  direct  items, 
are  given  in  a  single  battery  to  a  group 
of  behavior  problem  and  a  group  of  non- 
problem  school  children:  (a)  the  indirect 
test  scores  will  be  significantly  different 
from  the  direct  scores;  (b)  groups  of  be- 
havior problem  and  non-problem  chil- 
dren will  react  differently  to  the  indirect 
and  direct  tests;  and  (c)  presentation  of 
the  questions  in  three  different  orders— 
an  indirect-direct,  an  alternate,  and  a 
direct-indirect  order— will  have  a  distinct 
effect  on  the  degree  of  test  sophistication 
shown  by  the  children  in  their  responses. 

summary  of  the  experiment 

In  order  to  test  the  experimental 
hypothesis,  these  materials,  subjects,  and 
experimental  design  were  employed: 

Materials.  A  basic  personality  test 
form  of  36  items  was  constructed  by  tak- 
ing the  most  frequently  used  questions 
on  eight  standard  personality  question- 
naires for  children.  Two  direct  forms  of 
this  test  were  made:  (1)  with  all  36  ques- 
tions asking  about  a  negative  symptom 
(e.g.,  "I  cry  VERY  OFTEN  PRETTY 
OFTEN  SELDOM  NEVER.");  (2) 
with  all  36  questions  asking  about  a  posi- 
tive symptom  (e.g.,  "I  am  free  from  cry- 
ing VERY  OFTEN  PRETTY  OFTEN 
SELDOM  NEVER.").  Two  indirect 
forms  of  the  same  36  questions  were 
constructed:  (1)  with  all  questions  ask- 
ing how  queer  the  respondents  con- 
sidered   other    children    who    possessed 


negative  symptoms  (e.g.,  "Children  who 
often  cry  are  VERY  QUEER  PRETTY 
QUEER  A  LIE  ILK  QUEER  \(>I 
AT  ALL  QUEER.");  (2)  with  all  ques- 
tions asking  the  respondents  how  many 
other  children  there  were  who  possessed 
negative  symptoms  (e.g.,  "VERY  MANY 
PRETTY  MANY  FEW  NO  children 
often  cry.").  In  addition,  the  Pintner 
Intermediate  Test  of  Mental  Ability  was 
administered  to  all  subjects. 

Subjects:  The  experimental  groups 
consisted  of  forty  behavior  problem  boys 
and  221  non-problem  boys  from  regular 
seventh  and  eighth  grade  classes  of  three 
New  York  City  junior  high  schools. 

Experimental  Design.  The  four  test 
forms  were  bound  together  in  sixteen- 
page  booklets,  each  page  containing  nine 
questions  from  one  of  the  tests.  The 
booklets  were  bound  so  that  the  tests 
could  be  administered,  in  a  single  setting, 
in  three  different  orders  of  presentation: 
(1)  an  indirect-direct  order— all  eight 
pages  of  the  indirect  questions  followed 
by  the  eight  pages  of  the  direct  ques- 
tions; (2)  an  alternate  order— a  page  of 
direct  questions  alternated  with  a  page 
of  indirect  ones;  and  (3)  a  direct-indirect 
order— all  eight  pages  of  the  direct  ques- 
tions followed  by  the  eight  pages  of  the 
indirect  ones.  The  booklets  were  dis- 
tributed so  that  one-third  of  the  problem 
and  one-third  of  the  non-problem  sub 
jects  received  each  order  ol  presenta- 
tion. Problem  and  non-problem  children 
were  tested  in  their  regular  home-room 
classes,  with  no  segregation  of  the  groups 
taking  place. 

CONCLUSIONS    V\D   IMPLICATIONS 

The  results  of  the  experiment  sub- 
stantially   support    its    hypothesis,    the 
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main  findings  being  as  follows: 

1.  The  mean  scores  of  the  direct  tests 
do  not  differ  significantly  from  each 
other;  but  those  of  the  indirect  tests 
do.  The  mean  stores  of  the  two  direct 
tests  differ  significantly  from  those  of 
both   incline  t   tests. 

S».  Intel  correlations  between  the  two 
direct  tests  are  reasonably  high;  and 
main  item  identities  exist  between  these 
two  tests.  Intercorrelations  between  the 
two  indirect  tests  are  fairly  low.  Inter- 
correlations between  the  direct  and  in- 
direct tests  are  not  too  high;  and  item 
identities  from  the  direct  to  the  indirect 
tests  are  not  too  many.  One  indirect 
test  (1-2)  is  much  more  closely  related  to 
the  direct  tests  than  is  the  other  in- 
direct test  (I-i). 

3.  The  subjects  give  significantly  more 
unfavorable  responses  to  certain  kinds 
of  questions,  especially  psychosomatic 
ones,  than  they  do  to  other  kinds, 
especially  home  and  social  adjustment 
items. 

4.  The  subjects  react  quite  differently 
to  questions  taken  out  of  the  test  context 
than  they  do  to  those  taken  as  part  of 
the  regular  test  battery. 

5.  Significant  differences  between 
problem  and  non-problem  subjects' 
mean  scores  are  evident  only  for  one  of 
the  direct  tests  (D-2)  and  not  for  the 
other  direct  test  or  for  either  of  the 
direct  tests.  Observed  differences  be- 
tween problem  and  non-problem  groups 
of  subjects  are  probably  atributable  to 
the  small  number  and  non-homogeneous 
subgrouping  of  the  problem  children 
and  are  therefore  to  be  cautiously 
viewed  and  interpreted. 

6.  Few  items  discriminate  significantly 
between  the  problem  and  non-problem 
groups  on  any  of  the  four  tests,  though 
the  direct  tests  contain  more  item  dis- 
criminations than  do  the  indirect.  Dif- 


ferent specific  items  prove  discriminating 
on  the  direct  and  indirect  tests.  The 
items  which  discriminate  best  between 
problem  and  non-problem  groups  are 
largely  concerned  with  school  activities. 

7.  The  order  of  presentation  of  the 
direct  and  the  indirect  test  questions 
has  a  significant  effect  upon  average  test 
scores,  test  score  intercorrelations,  item 
discriminations  between  problem  and 
non-problem  groups,  biserial  correla- 
tions between  total  test  and  item  scores, 
and  percentages  of  subjects  responding 
unfavorably  to  different  kinds  of  ques- 
tions. 

The  main  implications  to  be  drawn 
from  the  experiment  are  presented  be- 
low. In  making  them,  it  must  be  em- 
phasized that  the  present  study,  being 
based  on  the  conventional  types  of  ques- 
tionnaires which  are  now  widely  used, 
is  doubtlessly  limited  by  some  of  the 
defects  which  are  inherent  in  such  tests. 
A  particular  limitation  of  the  study  is 
its  use,  for  its  problem  subjects,  of  a 
smaller  group  of  children  than  would 
have  been  statistically  desirable.  Con- 
sequently, the  following  conclusions 
and  implications  must  be  taken  as  tenta- 
tive, and  as  requiring  for  their  final  con- 
firmation considerable  further  experi- 
mentation. 

1 .  The  order  of  presentation  of  indirect 
and  direct  personality  questions  ad- 
ministered in  a  single  test  battery  may 
decisively  effect  test  results.  If  test-makers 
are  to  construct  instruments  that  will 
encourage  the  least  sophisticated,  most 
truthful  responses  by  their  subjects,  they 
must  make  certain  that  all  indirect 
items  are  printed  before  the  direct  ones. 

2.  Phrasing  is  an  important  considera- 
tion in  the  construction  of  personality 
test  items.  Each  mode  of  indirectly 
phrasing  a  schedule  of  questions  may  re- 
sult  in    the   creation   of   an    instrument 
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significantly  different  from  that  of  other 
indirectly  phrased  questionnaires. 

3.  Indirect  personality  tests  seem  to 
differentiate  problem  from  non-problem 
groups  of  children  no  better  than  do 
direct  test  forms.  Certain  specific  ques- 
tions, however,  are  more  discriminative 
when  indirectly  than  directly  phrased, 
so  that  optimum  results  may  possibly  be 
obtained  by  combining  both  kinds  of 
questions  in  a  single  test. 


4.  The  kinds  ol  questions  which  ;i])- 
pear  best  to  distinguish  problem  from 
non-problem  groups  of  children  are 
those  specifically  concerned  with  school 
activities. 

5.  Much  more  experimental  work 
must  be  done  with  personality  question- 
naires before  they  can  be  expected  to  be 
satisfactory  instruments  lor  the  diagnosis 
of  behavior  problem  difficulties. 
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APPENDIX 
I  111    TESTS  AND  INSTRUCTIONS  EMPLOYED  IN    THE  EXPERIMENT 

Instructions 

(To  he  road  aloud   to  the  Class) 

Do  not  open  your  booklet  until  you  are  given  the  signal  to  do  so.  We  would  like  your  help 
in  finding  out  what  school  children  think  about  themselves  and  what  they  think  about  other 
children.  Phase  answer  the  questions  as  honestly  as  you  can,  so  that  we  may  find  out  what 
school  children,  including  yourself,  are  really  like.  Of  course,  this  is  not  a  test.  These  questions 
have  nothing  to  do  with  your  school  work.  Your  teacher  will  not  see  them.  You  will  not  be 
marked  on  them. 

Look  at  the  top  of  the  page  in  front  of  you.  Fill  it  out  now.  Write  in  your  name.  Write 
today's  date.  Write  down  how  old  von  were  at  your  last  birthday.  Circle  whether  you  are  a 
boy  or  a  girl.  Put  down  your  official  class.  Write  the  number  of  this  school. 

Do  not  open  this  booklet  until  you  are  given  the  signal  to  go.  When  you  do  open  it,  you 
will  find  that  it  contains  three  different  kinds  of  questions.  Samples  of  each  one  of  these 
three  kinds  of  questions  are  printed  on  the  page  before  you.  You  may  now  look  at  these 
sample  questions.  First  look  at  Question  1.  Read  it  to  yourself  as  I  read  it  to  you.  It  says:  I 
oct  angry  VERY  OFTEN  PRETTY  OFTEN  SELDOM  NEVER."  This  question  asks 
something  about  yourself.  Answer  it  truthfully  about  yourself.  Some  of  you  may  get  angry 
VERY  OFTEN,  some  of  you  PRETTY  OFTEN,  some  SELDOM,  and  some  of  you  NEVER. 
One  of  these  descriptions  certainly  fits  you.  So  draw  a  circle  around  the  one  which  does 
describe  how  angry  you  get.  Remember:  only  draw  a  circle  around  ONE  answer  for  each 
question. 

Now  look  at  Question  2.  This  question  reads:  "VERY  MANY  PRETTY  MANY  FEW 
NO  children  are  afraid  of  high  places."  It  asks  about  other  children  whom  you  know.  We 
want  your  opinion  about  how  many  of  these  other  children  are  afraid  of  high  places.  Again, 
circle  only  one  of  the  four  answers  to  this  kind  of  question. 

Now  look  at  Question  3,  which  reads:  "Children  who  often  day-dream  are  VERY  QUEER 
PRETTY  QUEER  A  LITTLE  QUEER  NOT  AT  ALL  QUEER."  This  question,  again, 
asks  about  other  children.  We  want  your  opinion  about  children  who  often  day-dream,  and 
want  you  to  let  us  know  if  you  personally  think  they  are  VERY  QUEER,  A  LITTLE 
QUEER,  or  NOT  AT  ALL  QUEER.  This  is  not  a  test,  remember,  and  you  will  not  be 
marked  on  it. 

You  can  see,  then,  that  there  are  three  different  kinds  of  things  you  are  to  do  in  this 
booklet.  One  kind  of  question  asks  about  yourself,  and  the  other  two  kinds  ask  your  opinions 
about  other  children.  In  all  three  kinds  of  questions,  though,  you  are  to  draw  a  circle  around 
only  ONE  of  the  four  possible  answers.  Carefully  read  each  question  and  answer  it  as 
honestly  as  you  can.  Do  not  skip  any  of  them.  The  booklet  has  sixteen  pages  in  it,  but  it 
should  not  take  too  long  to  answer  all  its  questions.  There  is  no  speed  limit,  but  work  as 
fast  as  you  can. 

Your  truthful  answers  on  this  booklet  will  be  a  great  help  to  us  in  finding  out  what  school 
children  are  really  like,  so  be  sure  to  answer  every  single  question  carefully.  This  has  nothing 
to  do  with  your  school  work,  and  your  teacher  will  not  even  look  at  your  papers. 

Now  open  the  booklet  and  make  sure  that  there  are  exactly  sixteen  white  pages  of  ques- 
tions. If  there  are,  go  right  ahead  and  answer  all  the  questions.  If  there  are  not,  ask  your 
teacher  for  a  new  booklet. 

Test  D-i  Questions 

I  would  rather  play  by  myself  than  play 

with  other  children  aery  often         pretty  often         seldom         never 

I  find  it  hard  to  make  friends  with  other 

children  very  often         pretty  often         seldom         never 
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I  cry 

I  feel  like  running  away  from  home 

I  find  it  hard  to  do  my  school  work  right 

I  am  afraid  of  the  dark 

I  feel  ill  or  weak 

I  am  afraid  of  being  followed  when  I  am 

walking  on  the  street 
Other  members  of  my  family  are  treated 

better  than  I  am  treated 

1  find  school  a  hard  place  to  get  along  in 
Other  children  refuse  to  play  with  me 
I  find  it  hard  to  speak  out  in  class 

recitations 
I  feel  that  my  teachers  treat  me  badly  or 

unfairly 
My  feelings  are  easily  hurt 
I  feel  that  nobody  really  loves  me 
I  feel  tired  during  the  daytime 
I  find  home  a  hard  place  to  get  along  in 
I  feel  tired  when  I  wake  up  in  the 

morning 

I  am  afraid  during  a  thunderstorm 

I  have  headaches 

I  am  bothered  with  pains  in  some  part 

of  my  body 
I  am  afraid  of  high  places 
I  day-dream 
I  get  angry 
I  have  the  habit  of  twitching  my  neck 

or  face 
I  dislike  school 
People  find  fault  with  me 

My  parents  blame  me  or  find  fault 

with  me 
I  bite  my  nails 
I  am  afraid  of  fire 

I  have  horrible  dreams  while  sleeping 
I  find  it  hard  to  become  a  leader  in 

games  with  other  children 
I  become  gloomy  or  sad 
I  become  excited 

I  stutter  or  stammer  when  talking 
I  am  afraid  of  my  father 


VERY  OFTEN 
VERY  OFTEN- 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 

VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 


VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 

VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 
VERY  OFTEN 


PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OF  1 1  N 
PRETTY  OFTEN 

PRETTY  OFTEN 

PRETTY  OFTEN 

PRETTY  OFTEN- 
PRETTY  OFTEN 

PRETTY  OFTEN 

PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN- 
PRETTY  OFTEN 

PRETTY  OFTEN 

PRETTY  OFTEN 
PRETTY  OFTEN 

PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 

PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 


PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 

PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 
PRETTY  OFTEN 


SELDOM 

NEVER 

si  1  DOM 

NEVER 

si  1  l>OM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NE\  1  K 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVES 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

N  EVER 

SELDOM 

NEVER 

SET  DOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

N  EVER 

SELDOM 

N  EVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

NEVER 

SELDOM 

N  EVER 

SELDOM 

N  EVER 

SELDOM 

Nl  \  >  R 

SELDOM 

NEVER 

I  find  it  easy  to  speak  out  in  class 

recitations 
I  am  unafraid  of  my  father 
I  feel  well  and  strong 
I  am  unafraid  of  fire 
I  am  happy 
I  remain  calm  and  unexcited 


Test  D-2  Questions 


VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NT\  I  K 

VERY  OFTEN 

PRETTY  OFTEN 

si  1  DOM 

N  EVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

N  EVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

N  EVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 
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Other  children  are  willing  to  play  with 

me 
I  find  school  an  easy  place  to  get  along  in 
I  .mi  treated  as  well  as  other  members 

of  my  family  are  treated 

I  like  school 

I  am  free  from  crying 

I  keep  from  biting  mv  nails 

People  praise  me 

I  am  free  from  anger 

I  am  unafraid  of  the  dark 

I  am  free  from  feelings  of  wanting  to  run 

away  from  home 
I  am  free  from  having  my  feelings  easily 

hurt 
I  am  unafraid  during  a  thunderstorm 

I  find  home  an  easy  place  to  get  along  in 
I  feel  very  rested  when  I  wake  up  in  the 

morning 
I  am  free  from  day-dreaming 
I  am  free  from  pains  in  some  part  of 

my  body 
1  find  it  easy  to  become  a  leader  in 

games  with  other  children 
I  am  unafraid  of  high  places 
I  am  free  from  stuttering  or  stammering 

when  talking 
My  parents  praise  me 
While  sleeping  I  am  free  from  horrible 

dreams 

I  am  free  from  the  habit  of  twitching 

my  neck  or  face 
I  find  it  easy  to  make  friends  with  other 

children 
I  am  free  from  having  headaches 
I  feel  full  of  energy  during  the  day 
I  feel  that  my  teachers  treat  me  well 

and  fairly 
I  find  it  easy  to  do  my  school  work  right 
I  have  no  fear  of  being  followed  when  I 

am  walking  on  the  street 
I  feel  that  people  really  love  me 
I  would  rather  play  with  other  children 

than  play  by  myself 

Test  I-i  Questions 
Children  who  feel  tired  when  they  wake  in  the  morning  are 

VERY  QUEER 

Children  who  are  afraid  of  fire  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  find  it  hard  to  make  friends  with  other  children  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  feel  ill  or  weak  are 


VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY'  OFTEN 

SELDOM 

NEVER 

VERY  OFTEN 

PRETTY  OFTEN 

SELDOM 

NEVER 

A  LITTLE  QUEER 


NOT  AT  ALL  QUEER 


NOT  AT  ALL  QUEER 


NOT  AT  ALL  QUEER 


VERY  QUEER 


PRETTY  QUEER 


A  LITTLE  QUEER     NOT  AT  ALL  QUEER 
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Children  who  feel  that  other  members  of  their  family  are  treated  better  than  they 
are  treated  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER  NOT  AT  ALL  QUI  I  R 

Children  who  dislike  school  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER     NO  I  A  I  ALL  QUEER 

Children  whom  other  children  refuse  to  play  with  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER     NOT  AT  ALL  QUI  1  R 

Children  who  find  it  hard  to  speak  out  in  class  recitations  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER  NOT  AT  ALL  Ql  1  I  l< 

Children  who  would  rather  play  by  themselves  than  play  with  other  children  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER     NOT  AT  ALL  Ql  I  !  R 


Children  who  often  have  headaches  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  become  gloomy  or  sad  are 

\T.RY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  are  afraid  of  their  father  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  feel  that  people  often  find  fault  with  them  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  are  bothered  with  pains  in  some  part  of  their  body  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  feel  that  nobody  really  loves  them  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  are  afraid  of  being  followed  when  walking  on  the  street  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  feel  like  running  away  from  home  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  find  it  hard  to  become  a  leader  in  games  with  other  children  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER     NOT  AT  ALL  QUEER 


NOT  AT  ALL  Ql  I  1  R 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  Ql  1  I  R 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 


Children  who  often  cry  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER 

Children  who  are  afraid  of  high  places  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  find  it  hard  to  do  their  school  work  right  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  find  school  a  hard  place  to  get  along  in  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  get  angry  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  feel  that  their  parents  blame  them  or  find  much  fault  with  them  are 

VERY  QUEER     PRETTY  QUEER     A  LITTLE  QUEER     NOT  AT  ALL  QUEER 

Children  who  often  become  excited  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  have  horrible  dreams  while  sleeping  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  find  home  a  hard  place  to  get  along  in  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  often  day-dream  are 

VERY  QUEER  PRETTY  QUEER 

Children  who  are  afraid  of  the  dark  are 

VERY  QUEER  PRETTY  QUEER 

Children  who  are  afraid  during  a  thunder-storm  are 

VERY  QUEER  PRETTY  QUEER 

Children  who  bite  their  nails  are 

VERY  QUEER  PRETTY  QUEER 


A  LITTLE  QUEER 
A  LITTLE  QUI  IK 
A  LITTLE  QUEER 
A  LITTLE  QUEER 


NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  AI  I  QUE!  K 
NOT  AT  ALL  QUEER 


NOT  AT  ALL  Qt  I  I  R 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  Ql  1  I  K 
NOT  AT  ALL  QUEER 
NO!  VT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
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Children  who  often  feel  tired  during  the  daytime  are 

VERY  QUEER  PRETTY  QUEER  A  UTILE  QUEER 

Children  whose  feelings  arc  easily  hurt  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  feel  that  their  teachers  treat  them  badly  or  unfairly  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  stutter  or  stammer  when  talking  are 

VERY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Children  who  have  the  habit  of  twitching  their  neck  or  face  are 

\  FRY  QUEER  PRETTY  QUEER  A  LITTLE  QUEER 

Test  1-2  Questions 


NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 
NOT  AT  ALL  QUEER 


\  1  RY   M  WY 

PR]  n  y  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

\  1  KY  MANY 

PR]  (TYM  \NY 

FEW 

NO 

\  IKY  MANY 

PRETTY  MANY 

FEW 

NO 

\  !  RY  MANY 

PRETTY  MANY 

FEW- 

NO 

\  !  RY   M  \NY 

PRETTY  MANY 

FEW 

NO 

VERY  M  WY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

\  1  R1*    MANY 

PRF.T1Y  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

\  ERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

\  1  RY  MAN! 

PRETTY  MANY 

FEW 

NO 

VERY  MANY- 

PRETTY  MANY 

FEW- 

NO 

MR  Y  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY- 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY- 

PRETTY  MANY 

FEW 

NO 

VERY  MANY- 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW- 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

children  feel  tired  when  they  wake  up  in  the 

morning, 
children  bite  their  nails, 
children  often  get  angry, 
children  are  afraid  of  fire, 
children   find  it  hard  to  speak  out  in  class 

recitations, 
children  often  feel  tired  during  the  daytime, 
children  often  become  gloomy  or  sad. 
children  are  afraid  of  high  places, 
children  are  afraid  of  being  followed  when 

they  are  walking  on  the  street. 

children   feel  that  their  teachers  treat  them 

badly  or  unfairly, 
children  are  bothered  with  pains  in  some  part 

of  their  body, 
children    would    rather    play   by    themselves 

than  play  with  other  children, 
children  find  home  a  hard  place  to  get  along 

in. 
children  often  day-dream, 
children  are  afraid  of  their  fathers, 
children   find  it  hard  to  make  friends  with 

other  children, 
children  have  other  children  refuse  to  play 

with  them, 
children  find  it  hard  to  do  their  school  work 

right. 

children   have   the  habit  of  twitching   their 

neck  or  face, 
children  feel  that  nobody  really  loves  them, 
children  often  feel  like  running  away  from 

home, 
children  often  feel  ill  or  weak, 
children  often  cry. 
children   often   have   horrible   dreams   while 

sleeping, 
children  find  school  a  hard  place  to  get  along 

in. 
children    feel    that    other   members   of   their 

family   are    treated    better   than    they   are 

treated. 
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VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETIY'  MANY 

FEW 

NO 

VERY  MANY 

PRETIY  MANY 

FEW 

NO 

VERY  MANY 

PRETrY  MANY 

FEW 

NO 

VERY  MANY 

PRETIY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

VERY  MANY 

PRETTY  MANY 

FEW 

NO 

children  find  it  hard  to  become  a  leader  in 
games  with  other  children. 

children  feel  that  their  parents  blame  them 

or  find  much  fault  with  them, 
children  stutter  or  stammer  when  talking. 
children    feel    that   people   find   much   fault 

with  them, 
children  often  become  excited, 
children  are  afraid  of  the  dark, 
children  dislike  school. 

children  are  afraid  during  a  thunder-storm, 
children  often  have  headaches, 
children's  feelings  are  easily  hurt. 
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